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Description 

IMAGE PROCESSING SYSTEM, IMAGE 
PROCESSING METHOD, AND MEDIUM 
HAVING AN IMAGE PROCESSING 
CONTROL PROGRAM RECORDED 

THEREON 
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BACKGROUND OF THE INVENTION 

[0002] 1, Field of the Invention 

[0003] The present invention relates to an image processing sys- 
tem, an image processing method, and a medium having 
an image processing control program recorded thereon. 
Particularly, the invention is concerned with an image pro- 
cessing system and method for executing an optimum 
image processing with use of a computer, as well as a 
medium having an image processing control program 
recorded thereon. 

[0004] 2. Description of the Prior Art 

[0005] Heretofore, for image data of natural pictures such as 
photographs and image data of drawing type unnatural 
pictures, there have been known software such as pho- 
toretouch software for performing various effect process- 
ing. In such softwares, image data are displayed on a dis- 
play or the like, to which an operator applies a desired 
processing to form nice-looking image data. 

[0006] For example, in the case of reading a photograph through 



a scanner to obtain electronic image data, it is sometimes 
desired to liighliglit tlie contrast and vividness of the im- 
age. 

[0007] As a metliod for enlarging the contrast of image data 
there is known the method disclosed in Japanese Patent 
Publication No. 66318/95, in which the relation the lumi- 
nance y before transformation and the luminance y' after 
transformation are correlated with each other according to 
the following expression (1), and the luminance of image 
data is transformed on the basis of parameter 'a' or 'b' se- 
lected by an operator, whereby there is obtained a con- 
trast-highlighted image from image data weak in con- 
trast: 

[0008] y'=ay+ b... (1) 

[0009] On the other hand, as a method for highlighting satura- 
tion to highlight the vividness there is known a method 
wherein the value of a desired color component is in- 
creased in the case where the color components of image 
data are represented in terms of gradation data of red (R), 
green (G) and blue (B). 

[0010] More particularly, when the gradation data is in the range 
of 0 to 255, 20 is always added to the gradation data of 
red to make red color more vivid, or 20 is always added to 



the gradation data of blue to make blue color more vivid. 

[0011] Thus, among various effect processing there is included 
one which transforms natural pictures into more beautiful 
pictures, while some images are not required to be re- 
touched, such as drawing type images. 

[0012] In the latter case, in an image processing system using 
the above conventional software, the operator is required 
to Judge the type of image on the display and select an 
appropriate effect processing from among various ones or 
judge the degree of the processing. Thus, it has been im- 
possible to automate this operation. 

[0013] For example, it has so far been impossible to judge auto- 
matically whether contrast should be highlighted or not or 
whether vividness should be highlighted or not. 

[0014] In highlighting contrast, the conventional methods merely 
adopt the technique of providing plural settings different 
in strength of contrast beforehand and then switching 
over from one to another. Thus, it has been impossible to 
automatically select the most preferred one corresponding 
to the actual image data. Particularly, when luminance is 
changed in accordance with the foregoing expression (1), 
the result is such that only brightness is highlighted in the 
case of an image which is bright as a whole, or only dark- 



ness is highlighted in the case of an image which is darl< 
as a whole. 

[0015] Needless to say, the strength of contrast can be adjusted 
in television for example, but the parameter 'b', namely 
offset, is uniform and it has been impossible to effect an 
optimum highlighting for each individual image. 

[0016] Also as to highlighting vividness, a man is required to 
Judge to what degree of highlighting is to be performed 
with respect to each image data, and it has been impossi- 
ble to select the best suited highlighting automatically. 

[0017] Further, as to not only the contrast enlarging processing 
but also the saturation highlighting processing, there is a 
tendency to increase in value of image data. Therefore, if 
saturation is to be highlighted after highlighting of con- 
trast, a loud image results. This is not desirable. 

[0018] In this connection, in the above conventional image pro- 
cessing system, if one is adjusted to an appropriate con- 
dition and thereafter the other is adjusted, the result of 
the previous adjustment goes wrong and thus it is difficult 
to make adjustment as desired. 

[0019] In U.S. Patent No.5,181,105 there is disclosed a technique 
of diminishing the contrast width of image data automati- 
cally in view of the point that a color space reproducible in 



a CRT display is larger than that in a video printer. In this 
case, all that is required is merely causing image data to 
enter the color space in the video printer which is known 
in advance. Thus, automation is feasible because only 
such reduction of the contrast width is needed irrespec- 
tive of the type of image. Such an automation has nothing 
to do with the foregoing effect processing that intends to 
make the most of the image quality of the image data. 
Thus, the application of the aforesaid technique is mean- 
ingless. 

SUMMARY OF THE INVENTION 

[0020] Accordingly, it is an object of the present invention to 

provide an image processing system, an image processing 
method, and a medium having an image processing con- 
trol program recorded thereon, capable of Judging the 
type of image automatically on the basis of image data 
and performing an optimum image processing. 

[0021] The image processing system according to the present in- 
vention comprises a number-of-color detecting means 
which inputs image data representing information of each 
of pixels resolved in a dot matrix form from an image and 
which detects the number of colors used while regarding 
information corresponding to the luminance of each pixel 



as color, and an image discriminating means forjudging 
tlie type of image on tlie basis of tlie number of colors 
thus detected. 

[0022] jhe image processing method according to the present 
invention performs a predetermined image processing for 
image data which represents information of each of the 
pixels resolved in a dot matrix form from an image. The 
same method comprises inputting the said image data, 
detecting the number of colors used while regarding in- 
formation corresponding to the luminance of each pixel 
as color, and judging the type of image on the basis of the 
number of colors thus detected. 

[0023] Further, the medium according to the present invention 
has an image processing control program recorded 
thereon for inputting to a computer image data which 
represents information of each of the pixels resolved in a 
dot matrix form from an image and for performing a pre- 
determined processing. The said image processing control 
program includes the step of inputting the said image 
data and detecting the number of colors used while re- 
garding information corresponding to the luminance of 
each pixel as color and the step of judging the type of im- 
age on the basis of the number of colors thus detected. 



[0024] In the present invention thus constructed, when the num- 
ber-of-color detecting means detects image data which 
represents information of each pixel resolved in a dot ma- 
trix form from an image, it detects the number of colors 
used while regarding information corresponding to the lu- 
minance of each pixel as color, and the image discrimi- 
nating means judges the type of image on the basis of the 
number of colors thus detected. 

[0025] The number of colors used differs depending on the type 
of image. For example, in the case of a natural picture 
such as a photograph, even if the object to be pho- 
tographed is of the same blue color, plural colors are de- 
tected due to a shadow and thus a considerable number 
of colors may be used. On the other hand, in the case of a 
drawing type data or a business graph, a limit is encoun- 
tered in the number of colors used because it has been 
drawn with colors designated by an operator. 

[0026] According to the present invention, from the standpoint of 
the number of colors used, it is possible to judge the type 
of image automatically on the basis of image data. Need- 
less to say, images including a large number of colors 
used correspond to natural pictures. If the number of col- 
ors used is large, the image discriminating means judges 



that the image concerned is a natural picture, while if the 
number of color used is small, the image discriminating 
means judges that the image concerned is a business 
graph or the like. In connection with this image data, in- 
formation corresponding to each pixel represents of color 
directly or indirectly. Image data covers component values 
of plural elementary colors, coordinate values of a known 
absolute color space, and the brightness of a monotone. 
[0027] According to one method for detecting the number of col- 
ors used, actual colors of matrix pixels are judged and to- 
taled in a histogram form for detection. However, in the 
sense of judging the type of image, an exact value is not 
always needed. 

[0028] On the other hand, each color and luminance are corre- 
lated with each other, and although plural colors corre- 
spond to a single luminance value, there does not occur 
that the number of luminance values is large and that of 
color is small. Besides, it is actually inconceivable that an 
image is constructed of only colors of the same luminance 
value. Thus, it can be said that a rough tendency as to 
whether the number of colors used is large or not can be 
judged in terms of luminance values. 

[0029] According to the present invention, therefore, by replac- 



ing the number of colors used with luminance, it becomes 
easy to effect totaling even when the number of repro- 
ducible colors is very large. Thus, the 'number of colors 
used' as referred to herein is of a broad concept corre- 
sponding to the number of colors. 
[0030] In totaling luminance values, image data sometimes in- 
clude luminance data directly or may include luminance 
data only indirectly. In the case of direct luminance data, 
it suffices to make transformation thereof, or even in the 
case of indirect luminance data, it suffices to first trans- 
form it into luminance data and thereafter performs a 
predetermined luminance transformation. However, it is 
not that the transformation of luminance must be ex- 
tremely accurate, but it can be said that a rough transfor- 
mation will do. 

[0031] Therefore, it is another object of the present invention to 
provide an image processing system capable of obtaining 
luminance more easily even when image data does not in- 
clude any element that represents luminance directly. 

[0032] In the image processing system according to the present 
invention, in the case where image data is represented in 
terms of plural component values corresponding to lumi- 
nance, the number- of-color detecting means determines 



a luminance by weighted integration of the said compo- 
nent values. 

[0033] Despite the merit of using luminance, if there is included 
no luminance component in the parameter of image data, 
it is necessary to make color transformation. Usually, a 
large number of processings are required for color trans- 
formation. In this connection, since a slight error is al- 
lowed at the time of utilizing luminance, in the case where 
image data is represented in terms of plural component 
values corresponding to luminance, luminance can be de- 
termined by weighted integration of the component values 
without requiring a large number of processes. For exam- 
ple, where image data is represented by the gradation 
data of RGB (red, green, blue), it can be said that the com- 
ponent values of red, green and blue each correspond to 
luminance. For this reason it can be said that a linear ad- 
dition of the component values represent luminance 
enough. Thus, this can be an extremely easy method for 
transformation. 

[0034] There also are available other methods for simplifying the 
detection of the number of colors used. For example, a 
lattice shape sporadic in each axial direction is considered 
in a three-dimensional color space, and it is assumed that 



each group represents one color. By so doing, it is no 
longer required to total a very large number of colors such 
as, for example, 256 gray scales in each axis. 

[0035] On the other hand, in the case of a large image data, even 
a mere totaling of the number of colors used requires a 
considerable volume of processing. In view of this point it 
is a further object of the present invention to provide an 
image processing system capable of totaling the number 
of colors used in a more simple manner. 

[0036] According to this image processing system, the number- 
of-color detecting means makes sampling of pixels sub- 
stantially uniformly from among all the pixels and the 
number of colors is detected on the basis of the image 
data concerning the sampled pixels. 

[0037] Since the number of colors used is not always required to 
be strict, it is not always necessary that the totaling of 
colors throughout the whole image data be conducted for 
all the pixels. 

[0038] Needless to say, although an error results from thinning- 
out, since it is not necessary to know an exact number of 
colors used, as mentioned above, it suffices to detect the 
number of colors with respect to pixels sampled uniformly 
from among all the pixels, whereby the number of pixels 



and hence the amount of processing are decreased. 
[0039] For ensuring a certain degree of randomness by such 
sampling it is necessary to ensure an appropriate sam- 
pling ratio. Since an image is planar, the associated image 
data are inevitably distributed in both longitudinal and 
transverse directions. But in determining a sampling ratio, 
by ensuring an appropriate number of pixels sampled at 
least on the shorter side, it becomes possible to maintain 
a sureness proportional to the sampling ratio to be deter- 
mined. 

[0040] On the other hand, the image data having been subjected 
to image processing can be applied not only to display but 
also to various other purposes, including printing. In 
printing, it is necessary to make transformation into a 
color space of printing ink different from that of the origi- 
nal image data. But in relation to the number of colors 
used, the type of image exerts a great influence on the 
amount of processing in such color transformation. In 
view of this point it is a further object of the present in- 
vention to provide an image processing system capable of 
selecting and executing an optimum color transformation 
processing according to the type of image. 

[0041] This image processing system comprises a pre-gray level 



transforming means provided witli a table liaving lattice 
points in a color-specification space of data to be trans- 
formed in association with colorimetric gradation data in a 
color-specification space of data after transformation, the 
pre-gray level transforming means performing a gray level 
transformation of colorimetric gradation data before 
transformation into those corresponding to the lattice 
points in the said table, then referring to the same table 
and reading the corresponding colorimetric gradation 
data for color transformation; an interpolating color 
transformation means capable of making color transfor- 
mation into corresponding colorimetric gradation data by 
interpolating calculation between lattice points in the 
above table and having a storage area capable of reading 
data at high speed and storing information on that color 
transformation, the interpolating color transformation 
means utilizing a cache for color transformation by an in- 
terpolating calculation in the case of information not 
stored in the said storage area; a color transformation se- 
lection control means which, in the case of a natural pic- 
ture having a large number of colors used in the image 
data, makes color transformation with use of the pre-gray 
level transforming means and which, in the case of an im- 



age having a small number of colors in the image data 
and being not a natural picture, makes color transforma- 
tion with use of the interpolating color transformation 
means. 

[0042] In the case of making color transformation between dif- 
ferent color-specification spaces, a table is provided in 
which gradation data in the color-specification space after 
transformation are associated with the lattice points in the 
like space before transformation, and corresponding data 
at predetermined lattice points are read out successively 
by referring to the said table, whereby it is made possible 
to effect transformation. However, if there are provided 
lattice points corresponding to all the gray scale values of 
the gradation data before transformation, the table be- 
comes too large. For this reason, for example by error dif- 
fusion, the pre-gray level transforming means makes a 
gray level transformation of the gradation data before 
transformation into gradation data corresponding to the 
lattice points in the table to thereby reduce the size of the 
table. Also as to the other gradation data than at the table 
lattice points, transformation can be done by interpolating 
calculation. In the case where the number of colors is 
large, the gray level transformation executed by error dif- 



fusion for example is cliaracteristic in that the amount of 
calculation is sure to become smaller. 
[0043] However, this method requires that the error diffusion 
processing must be performed for all the pixels. There- 
fore, in the case of an unnatural picture which is small in 
the number of colors used, the result obtained by a single 
transformation can be utilized many times. In such a case, 
the interpolating color transformation means utilizing a 
cache may be advantageous at a point of processing 
speed, from the function of repeating the processing of 
storing the result of color transformation in the high- 
speed readable storage area while performing an interpo- 
lating calculation and reading it from the storage area if 
necessary. 

[0044] According to the present invention, therefore, in the case 
of a natural picture using a large number of colors, it is 
possible to make color transformation at a minimum 
amount of calculation in accordance with the error diffu- 
sion technique for example, while in the case where the 
number of colors used is small, the result of transforma- 
tion can be utilized repeatedly by using a cache. Thus, the 
amount of processing can be kept to a minimum by a 
color transformation method adopted in accordance with 



the type of image. 

[0045] Judgment of the type of image can also be utilized for 
judgment as to whether edge highlighting is to be per- 
formed or not. It is therefore a further object of the 
present invention to provide an image processing system 
capable of judging whether edge highlighting is to be 
performed or not. 

[0046] This image processing system comprises a natural picture 
discriminating means which, in the case where the num- 
ber of colors judged on the above image date is a prede- 
termined number of colors or more, judges that the image 
data represents a natural picture, and an edge emphasiz- 
ing means which, when the image data is judged to be a 
natural picture by the natural picture discriminating 
means, determines a low-frequency component on the 
basis of a marginal pixel distribution with respect to each 
of the pixels which constitute the image data, and dimin- 
ishes the low-frequency component, thereby eventually 
making a modification so as to enhance the edge degree 
of each pixel. 

[0047] In the present invention constructed as above, when the 
number of colors judged on the image data is a predeter- 
mined number or more, the natural picture discriminating 



means Judges that the image data represents a natural 
picture. When this judgment is made by the natural pic- 
ture discriminating means, the edge highlighting means 
determines a low-frequency component on the basis of a 
marginal pixel distribution with respect to each of the 
pixels which constitute the image data, and diminishes 
the low-frequency component, thereby eventually making 
a modification so as to enhance the edge degree of each 
pixel. Thus, when the image data represents a natural pic- 
ture, there is performed an image processing for edge 
highlighting, while when it is an unnatural image, there is 
not performed edge highlighting. 

[0048] In the case of a photograph as an example, certain pho- 
tographing conditions afford a blurry image whose edge is 
blunt. To prevent this inconvenience, an image processing 
is sometimes conducted. More specifically, when the 
number of colors in image data detected by the number- 
of-color detecting means is large, it is judged that the im- 
age data represents a natural picture, and in this case 
there is performed an image processing for edge high- 
lighting. Needless to say, the degree of edge highlighting 
can be changed as necessary. 

[0049] It is not necessary that the automation based on image 



data be limited to tlie judgment of image type. For exam- 
ple, in the case of a natural picture, it is preferable that its 
contrast be adjustable automatically. It is therefore a fur- 
ther object of the present invention to provide an image 
processing system, an image processing method, and a 
medium having an image processing control program 
recorded thereon, capable of adjusting contrast automati- 
cally on the basis of image data. 

[0050] This image processing system comprises a luminance dis- 
tribution totaling means which inputs image data repre- 
senting information of each of pixels resolved in a matrix 
form from an image and which totals luminance values of 
the pixels into a luminance distribution as a whole on the 
basis of the luminance of each pixel, and an image data 
luminance transforming means which, when the thus- 
totaled luminance distribution is not widely dispersed in 
an effective luminance range of the image data, trans- 
forms the luminance information of each pixel in the im- 
age data so that the above luminance distribution covers 
widely the luminance range. 

[0051] The image processing method according to the present 

invention performs a predetermined image processing for 
image data which represents information of each of pixels 



resolved in a dot matrix form from an image. According to 
tiiis method, luminance values of the pixels are totaled 
into a luminance distribution as a whole on the basis of 
the luminance of each pixel, and when the thus-totaled 
luminance distribution is not widely dispersed in an effec- 
tive luminance range of the image data, the luminance in- 
formation of each pixel in the image data is transformed 
so that the luminance distribution widely covers the said 
luminance range. 
[0052] The medium according to the present invention has an 
image processing control program recorded thereon for 
inputting to a computer image data which represents in- 
formation of each of pixels, resolved in a dot matrix form 
from an image and for performing a predetermined image 
processing. The image processing control program com- 
prises the steps of inputting the image data and totaling 
luminance values of the pixels into a luminance distribu- 
tion as a whole, and when the thus-totaled luminance 
distribution is not widely dispersed in an effective lumi- 
nance range of the image data, transforming the lumi- 
nance information of each pixel in the image data so that 
the luminance distribution widely covers the said lumi- 
nance range. 



[0053] In the present invention tlius constructed, what is called 
the width of contrast in the image data can be digitized to 
some extent by determining a luminance distribution in 
the image data, and after the digitization can be done, the 
said distribution is enlarged correspondingly to a repro- 
ducible range. It is not that the digitization always re- 
quires concrete numerical values. In the course of pro- 
cessing, the data may be treated as a numerical value or 
as the magnitude of signal. 

[0054] Jo be more specific, for determining a related luminance 
distribution in handling image data of a predetermined 
image, the luminance distribution detecting means de- 
tects a luminance distribution of the image data in the 
unit of pixel. Then, using this detected luminance distri- 
bution, the image data luminance transforming means 
judges an amount of enlargement of the luminance distri- 
bution in an effective available range and makes transfor- 
mation of the image data. 

[0055] Thus, by obtaining a luminance distribution of the image 
data in the unit of pixel, it is made possible to judge the 
so-called contrast width ranging from the highest lumi- 
nance to the lowest luminance, and an enlargement ratio 
can be determined by comparison with the effective lumi- 



nance range. Therefore, after that, all that is required is 
merely enlarging the luminance distribution so as to af- 
ford the enlargement ratio. 

[0056] For enlarging the luminance distribution there may be 

adopted several methods insofar as the luminance distri- 
bution of image data is enlarged in an allowable range. In 
view of this point it is a further object of the present in- 
vention to provide a luminance distribution enlarging 
method and an example thereof. 

[0057] In the image processing system according to the present 
invention, the image data luminance transforming means 
compares a statistical width of the luminance distribution 
detected with the width of the aforesaid luminance range 
and determines an enlargeable degree as the enlargement 
ratio. At the same time, the image data luminance trans- 
forming means determines an adjustment value to make 
adjustment so that the upper and lower ends of the thus- 
enlarged luminance distribution are within the luminance 
range in question, and modifies the luminance of each 
pixel. As an example, given that the range of a repro- 
ducible luminance is y range, the luminance Y after trans- 
formation is obtained from the luminance y before trans- 
formation and a maximum value y max and a minimum 



value y min in the luminance distribution range in accor- 
dance with the following expression: 
[0058] Y = ay + b...(2)where, 

[0059] a = yrange/(y -y ) ...(3) 

max min 

[0060] b = -any ory -any ...(4) 

min range max 

[0061] In the above transformation expressions, Y = 0 if Y < 0 
and Y = y ifY>y 

range range 

[0062] According to the concept of the transforming method, 
both enlargement ratio and adjustment value are deter- 
mined and the luminance of each pixel is modified on the 
basis of those values. An example is a linear enlargement. 
Although the transformation expression itself is the same 
as the conventional one, a significance resides in that the 
parameter thereof is selected by the image data lumi- 
nance transforming means. Irrespective of the selection of 
b, Y=Oify=y and Y= y ify=y . And the lumi- 

min range max 

nance distribution expands uniformly within the y range 
which is a reproducible luminance range. The transforma- 
tion in this example is a linear transformation in a narrow 
sense and no limitation is placed thereon. It is also possi- 
ble to make a non-linear transformation in a broad sense. 
The transforming expression in question is merely one 



example and it goes without saying tliat even otiier trans- 
forming expressions of tlie same meaning may also be 
employed. 

[0063] According to the present invention thus constructed it is 
possible to enlarge the luminance distribution effectively 
within a predetermined range of gradation. 

[0064] However, a mere expansion of contrast may sometimes be 
unable to cope with the case where the whole is bright or 
dark. It is therefore a further object of the present inven- 
tion to adjust the whole brightness automatically. 

[0065] In the image processing system according to the present 
invention, the image data luminance transforming means 
determines a maximum distribution luminance of the lu- 
minance y before transformation and executesycorrection 
to change the luminance distribution so that the range to 
which the maximum distribution luminance gives a de- 
sired brightness, thereby obtaining the luminance Y after 
transformation. 

[0066] Forjudging whether image data is bright or not as a 

whole, the maximum distribution luminance of luminance 
y before transformation is utilized, and if the maximum 
distribution luminance is on the bright side, there is 
madeycorrection to render the whole rather dark, while if 



the maximum distribution luminance is on tlie darlc side, 
tliere is madeycorrection to render tlie whole rather 
bright. In this way the entire brightness is corrected auto- 
matically, which cannot be attained by only the highlight- 
ing of contrast. It is optional whether the maximum distri- 
bution luminance of luminance y before transformation is 
to be obtained in terms of median or in terms of a mean 
value. 

[0067] Thus, according to the present invention, the degree of 
brightness which cannot be adjusted by only the high- 
lighting of contrast is also adjustable. 

[0068] In transforming the luminance by any of various methods, 
it is also possible to first calculate and store the lumi- 
nance Y after transformation within the range of the lumi- 
nance y before transformation and then call up the corre- 
lation at the time of transformation. It is not impossible to 
calculate the luminance every time in accordance with 
transforming expressions, but the range of values which 
the luminance distribution can take has already been de- 
termined. Therefore, if the luminance Y after transforma- 
tion is calculated and stored beforehand on the basis of 
the luminance y before transformation, it becomes possi- 
ble to effect transformation by merely calling up the cor- 



relation when the transformation is to be made. 

[0069] When the luminance distribution of image data is reviewed 
statistically, it is proper to consider that the luminance 
distribution extends up to both-end positions in a repro- 
ducible luminance range although such a case is seldom 
encountered. It follows that both ends of an actual lumi- 
nance distribution always correspond to both ends in a 
reproducible luminance range. If both ends in question 
are adopted, the enlargement ratio substantially becomes 
equal to '1' and it is impossible to attain the expected ef- 
fect. It is therefore a further object of the present inven- 
tion to obtain a substantial luminance distribution range 
which is enlargeable. 

[0070] In the image processing system according to the present 
invention, the luminance distribution totaling means re- 
gards positions inside by a predetermined distribution ra- 
tio from the actual ends of a luminance distribution as 
end portions at the time of determining the above lumi- 
nance distribution. 

[0071] In the present invention thus constructed, by removing 
the predetermined distribution ratio at both ends, the 
skirt portion of an extremely reduced distribution is ig- 
nored moderately from the statistical point of view. 



Therefore, this range is used as a criterion in judging the 
degree of enlargement. 
[0072] The predetermined distribution ratio is not specially lim- 
ited if only it permits the skirt portion of an extremely re- 
duced distribution to be ignored. For example, it may be 
the number of pixels corresponding to a certain ratio of 
the total number of pixels, or the positions where the dis- 
tribution ratio is below a certain ratio may be regarded as 
end portions. 

[0073] Thus, according to the present invention it is possible to 
obtain a luminance distribution more effective for judg- 
ment. 

[0074] On the other hand, in an actual image there are both 

highlight portion and high-shadow portion, and the hu- 
man eyes are easy to sense a delicate difference in these 
portions. Therefore, if an intentional enlargement reach- 
ing the end portions in a reproducible luminance range is 
performed, the highlight portion gives rise to a bald white 
impression, while the high-shadow portion affords a solid 
black impression. 

[0075] It is therefore a further object of the present invention to 
provide an image processing system capable of improving 
the expressibility of such highlight and high-shadow por- 



tions. 

[0076] In this image processing system, a luminance distribution 
range to be enlarged is set inside the end portions of the 
actual reproducible range by a predetermined amount. By 
so doing, an intentional enlargement is no longer per- 
formed at both end portions. 

[0077] Thus, according to the present invention it is possible to 
prevent collapse of the highlight and high- shadow por- 
tions. 

[0078] In some case a narrow contrast is inevitable. For example, 
in an evening scene it is natural that the luminance distri- 
bution width should be narrow. If it is enlarged to a 
greater extent than necessary, a daytime scene will result. 
It is therefore a further object of the present invention to 
provide an image processing system capable of prevent- 
ing the impression of an image from being transformed to 
an unnatural degree. 

[0079] In this image processing system, a limit is imposed on the 
enlarging degree of luminance distribution. By so doing, 
image data whose narrow contrast is inevitable is pre- 
vented from being enlarged its luminance distribution to 
an excessive degree. Needless to say, the above phe- 
nomenon is not limited to evening scenes but can be ob- 



served also in other cases. The occurrence of such a phe- 
nomenon is prevented by placing a limitation on the en- 
larging degree of luminance distribution. 

[0080] Thus, according to the present invention it is possible to 
prevent excessive highlighting of contrast which would 
causes a change in atmosphere of the image concerned. 

[0081] If a luminance distribution is used to the maximum extent 
within its reproducible range, there no longer remains a 
margin for enlargement of the luminance distribution. 
However, if a limit is imposed on the degree of enlarge- 
ment, there remains a margin which permits enlargement 
of the luminance distribution. In other words, there re- 
mains freedom for selecting actual range to be used for 
the enlargement, and this selection may cause a change in 
atmosphere of image. It is therefore a further object of the 
present invention to provide an image processing system 
capable of automatically adjusting the margin which per- 
mits enlargement of a luminance distribution even when a 
limitation is placed on the degree of enlargement. 

[0082] In this image processing system, a corresponding position 
of a luminance distribution range before transformation 
and that after transformation in a reproducible range are 
maintained to prevent the center of the luminance distri- 



bution from being changed. 

[0083] The luminance distribution center of image in sucli a 

sense can be grasped in various ways. For example, the 
luminance distribution may be enlarged so that an en- 
largeable range ratio remaining at each of upper and 
lower ends of the luminance distribution range before 
transformation is retained also after transformation. 

[0084] Jo be more specific, since it suffices if only the center is 
retained, it can be said unnecessary to directly catch and 
hold the center. Conversely, an enlargeable range remain- 
ing at each of upper and lower ends of the luminance dis- 
tribution range before transformation is noted and the lu- 
minance distribution is enlarged so as to retain the ratio 
of that range also after transformation, thereby holding 
the center substantially. 

[0085] As another example, it is preferable that the automatic 
judgment based on image data be applied to judging the 
degree of highlighting at the time of highlighting the 
vividness of a natural picture. It is therefore a further ob- 
ject of the present invention to provide an image process- 
ing system, an image processing method and a medium 
having an image processing control program recorded 
thereon, capable of adjusting the vividness automatically 



on the basis of image data. 

[0086] This image processing system comprises a saturation dis- 
tribution totaling means wliich inputs image data repre- 
senting information of eacli of pixels resolved in a dot 
matrix form from an image and which then totals satura- 
tions of the pixels into a saturation distribution as a 
whole, a saturation transformation degree judging means 
forjudging a saturation transformation degree of image 
data from the saturation distribution obtained by the sat- 
uration distribution totaling means, and an image data 
saturation transforming means which modifies saturation 
information contained in the image data on the basis of 
the transformation degree thus judged and transforms the 
modified information into a new image data. 

[0087] The image processing method according to the present 
invention is for performing a predetermined image pro- 
cessing for image data which represents information of 
each of pixels resolved in a dot matrix form from an im- 
age. The method comprises totaling saturations of the 
pixels into a saturation distribution as a whole, then judg- 
ing a saturation transformation degree for the image data 
from the saturation distribution thus obtained, then mod- 
ifying saturation information contained in the image data 



on the basis of the transformation degree thus judged and 
transforming it into a new image data. 

[0088] Further, the medium according to the present invention 
has an image processing control program recorded 
thereon for inputting to a computer image data which 
represents information of each of the pixels resolved in a 
dot matrix form from an image and for performing a pre- 
determined image processing, the program comprising 
the steps of inputting the image data and totaling satura- 
tions of the pixels into a saturation distribution as a 
whole, judging a saturation transformation degree for the 
image data from the saturation distribution thus obtained, 
and modifying saturation information contained in the im- 
age data on the basis of the transformation degree thus 
judged and transforming it into a new image data. 

[0089] In the present invention constructed as above, after the 
saturation distribution totaling means has totaled satura- 
tions of the pixels in the image data into a saturation dis- 
tribution as a whole, the saturation transformation degree 
judging means judges a saturation transformation degree 
for the image data from the saturation distribution thus 
obtained, and the image data saturation transforming 
means transforms the image data in accordance with the 



transformation degree thus judged. That is, for each im- 
age, an optimum transformation degree is judged from a 
saturation distribution of image data and the image data 
is transformed on the basis thereof. 

[0090] According to the present invention, therefore, it is possi- 
ble to provide an image processing system wherein a sat- 
uration transformation best suited for each image can be 
done using a totaled saturation distribution. 

[0091] In totaling saturations of pixels in image data into a satu- 
ration distribution, if the image data has saturation pa- 
rameters, it suffices to total the parameters. Even in the 
case where the image data does not possess such param- 
eters, it is possible, for example, to make a color trans- 
formation from another color specification space for a 
color specification space having such a saturation param- 
eter and then total saturation parameters after the trans- 
formation. However, a considerable volume of processing 
is needed for the color transformation. It is therefore a 
further object of the present invention to total saturations 
without color transformation. 

[0092] In the image processing system according to the present 
invention, the saturation of each pixel is judged in accor- 
dance with the saturation of a warm color hue in color 



components. 

[0093] Although it is difficult to judge saturation without a 
change of color specification space, the human visual 
characteristic tends to regard the difference between a 
warm color hue and a non-warm color hue as being vivid, 
so it is relatively convenient to Judge saturation on the 
basis of such difference. 

[0094] Thus, according to the present invention it is possible to 
total saturations in accordance with the human visual 
characteristic. 

[0095] In the image processing system according to the present 
invention, saturation (= X) is represented by the following 
expression, provided the color components of image data 
are red (R), green (G) and blue (B): 

[0096] X = |G + B-2XR|...(5) 

[0097] In the color specification space of RGB which is often used 
in a computer, there occurs a saturation-free state upon 
coincidence of the components and in the other cases 
there occurs saturation. In this connection, it is possible 
to judge the difference from the saturation-free state and 
thereby determine saturation, but the use of the relation- 
ship |G+B-2xR| is preferred because the coincidence of 
the components affords a minimum value irrespective of 



component values, also because a maximum value is ob- 
tained in red and light blue monocolor and further be- 
cause also in the case of blue or green there is presented 
a relatively large saturation. Needless to say, based on the 
same way of thinking, a like simplicity is attained by: 

[0098] x'= |R+B-2xG| ...(6) 
[0099] XD= |G + R-2XB| ...(7) 

[0100] Judging from experimental results, however, the best re- 
sults are obtained by the expression (5). It can be said 
that this is supported by the human visual characteristic 
of recognizing vividness according to the saturation of a 
warm color hue. 

[0101] According to the present invention, therefore, in the case 
of adopting the color specification of RGB, it is possible to 
decrease the calculation volume and total saturations in a 
simple manner. 

[0102] Needless to say, as is the case with enlargement of lumi- 
nance distribution, it is not always necessary that the sat- 
uration distribution in an image be obtained with respect 
to all the pixels of image data, insofar as it is possible to 
total saturations of pixels in image data into a saturation 
distribution. No special limitation is placed on a concrete 



totaling method. 

[0103] In judging tlie degree of saturation transformation from 
the state of saturation distribution, there may be adopted 
any of various analyzing methods for analyzing the state 
of distribution concretely. Now, it is a further object of the 
present invention to give an example of a concrete ana- 
lyzing method for the state of distribution. 

[0104] In the image processing system according to the present 
invention, the saturation transformation degree judging 
means strengthens the degree of saturation highlighting 
in the case of a small saturation at a predetermined ratio 
from the upper end in the totaled saturation distribution, 
while when the said saturation is large, it weakens the 
saturation highlighting degree, thereby judging the de- 
gree of saturation transformation. 

[0105] Thus, in judging the state of saturation distribution, a 

portion of the distribution is taken out from a large satu- 
ration side, then a judgment is made as to whether the 
saturation of that portion is large or small, and the entire 
saturation is detected from the result of the judgment, 
then the saturation is highlighted if it is small, while if the 
saturation is sufficiently large, it is judged that highlight- 
ing is not necessary. Needless to say, there may be 



adopted any other method, for example another statistical 
method, to judge the saturation tendency of the entire 
image data. Further, image data may be grasped partially, 
not wholly. 

[0106] Thus, according to the present invention, judging the de- 
gree of saturation transformation at a predetermined ratio 
from the upper end in the totaled saturation distribution 
facilitates the judging. 

[0107] There also are various other concrete methods for trans- 
forming the saturation of image data. For example, the 
transformation of saturation may be performed by radial 
displacement according to the foregoing degree of trans- 
formation within the Luv space in the standard calorimet- 
ric system. 

[0108] To be more specific, if image data has a saturation pa- 
rameter, this parameter may be transformed, but in the 
Luv space as the standard calorimetric system having a 
parameter of luminance or lightness and a parameter of 
hue in a plane coordinate system with respect to each lu- 
minance, the radial direction corresponds to saturation. In 
the Luv space, therefore, the transformation of saturation 
is performed by radial displacement. 

[0109] The reason why the Luv space is adopted is that lumi- 



nance is independent and tliat tlie displacement of satu- 
ration exerts no influence on luminance. In the case of us- 
ing the Luv space, however, it is necessary to make trans- 
formation if there is no correspondence of the original 
image data thereto. 
[0110] On the other hand, in the case where image data is ex- 
pressed by equal hue components, as is often the case 
with image data, there may be adopted a method wherein 
components values exclusive of a non-saturation compo- 
nent are shifted according to the degree of transforma- 
tion. 

[om] In the case where image data is represented by compo- 
nent values of plural, nearly equal hue components like 
RGB, it can be said that there is a saturation-free compo- 
nent. It follows that the component values exclusive of the 
saturation-free component exert an influence on satura- 
tion. In this case, saturation is transformed by displacing 
the component values. 

[0112] As an example of coping with such saturation-free com- 
ponent there may be adopted a method wherein a mini- 
mum component value in plural hue components is sub- 
tracted from the other component values and the differ- 
ential value thus obtained is increased or decreased to ef- 



feet saturation transformation. 

[0113] Of the plural color components, the minimum component 
value is also contained in the other color components, and 
such minimum component values are merely combined 
together and constitute a saturation- free gray. Thus, the 
differential value based on the other colors and exceeding 
the minimum component value exerts an influence on 
saturation, and by increasing or decreasing the differential 
value there is performed transformation of saturation. 

[0114] According to another example, a corresponding value of 
luminance is subtracted from each component value and 
the differential value obtained is increased or decreased 
to perform the transformation of saturation. A mere dis- 
placement of component values exclusive of a saturation- 
free component will cause a change in luminance. For this 
reason, if a corresponding value of luminance is sub- 
tracted beforehand from each component value and the 
transformation of saturation is performed by increasing or 
decreasing the resulting differential value, it becomes 
possible to store luminance. 

[0115] As to the luminance or lightness and the saturation are in 
a relation such that the color specification space assumes 
an inverted cone shape up to a certain range. It can be 



said that the lower the luminance, the larger the compo- 
nent values of hue. In such a case, if an attempt is made 
to apply a transformation degree proportional to a small 
value of saturation, there is a fear of breaking through the 
conical color specification space. Therefore, when the lu- 
minance is low, the transformation degree of saturation 
may be weakened to prevent the occurrence of such in- 
convenience. 

[01 1 6] The transformation degree of saturation is judged by any 
of the foregoing various methods. However, if saturation 
is highlighted to a greater extent than necessary even 
when it is weak, there will not be obtained good results. 

[0117] As to both saturation highlighting process and contrast 
enlarging process, there is a tendency to increase the 
value of image data. Therefore, if contrast is highlighted 
after highlighting of saturation, a loud image will result. 
This is not desirable. 

[0118] However, since both processings are conducted each indi- 
vidually, one is first adjusted to an appropriate state and 
thereafter the other is adjusted, so it is unavoidable that 
the result of the previous adjustment will go wrong. 

[0119] It is therefore a further object of the present invention to 
provide an image processing system, an image processing 



method and a medium having an image processing con- 
trol program, capable of obtaining a desired image more 
easily. 

[0120] The image processing system according to the present in- 
vention inputs image data which represents information of 
each of thepixels resolved in a dot matrix form from an 
image and then executes a predetermined image process- 
ing. The image processing system is provided with a con- 
trast enlarging means for enlarging the luminance distri- 
bution in the image data, a saturation highlighting means 
for highlighting saturation in the image data, and a high- 
lighting process suppressing means for suppressing the 
luminance distribution enlarging operation and the satu- 
ration highlighting operation with respect to each other. 

[0121] The image processing method according to the present 
invention comprises inputting image data which repre- 
sents information of each of pixels resolved in a dot ma- 
trix form from an image, enlarging the luminance distri- 
bution in the image data and highlighting saturation in 
the image data. The luminance distribution enlarging op- 
eration and the saturation highlighting operation are per- 
formed in a correlated manner so as to suppress each 
other. 



[0122] Further, the medium according to the present invention 
has an image processing control program recorded 
therein, the said program comprising the steps of in- 
putting image data which represents information of each 
of the pixels resolved in a dot matrix form from an image, 
enlarging the luminance distribution in the image data, 
highlighting saturation in the image data, and suppress- 
ing the luminance distribution enlarging operation and 
the saturation highlighting operation with respect to each 
other. 

[0123] In the present invention thus constructed, the contrast 
enlarging means enlarges the luminance distribution in 
the image data, while the saturation highlighting means 
highlights the saturation of each pixel. In this connection, 
the highlighting process suppressing means suppresses 
the luminance distribution enlarging operation and the 
saturation highlighting operation with respect to each 
other. 

[0124] Thus, when contrast is enlarged by the contrast enlarging 
means and saturation is highlighted by the saturation 
highlighting means, the highlighting process suppressing 
means suppresses both highlighting operations with re- 
spect to each other to prevent a synergistic effect of af- 



fording a loud image. l\/loreover, even if after one adjust- 
ment there is made tlie otiier adjustment, it is possible to 
keep the previous adjustment effective. 

[0125] The highlighting process suppressing means is not spe- 
cially limited if only it can suppress both luminance distri- 
bution enlarging operation and saturation highlighting 
operation with respect to each other. In this connection, 
the luminance distribution enlarging operation can be 
done by various methods and this is also true of the satu- 
ration highlighting operation. Therefore, concrete meth- 
ods may be selected suitably according to the contrast 
enlarging means and saturation highlighting means 
adopted. In suppressing both operations with respect to 
each other, it is not always necessary to suppress the two 
mutually. The suppressing method may be such that the 
suppressing process is applied from one to the other but 
is not applied from the other to one. By so doing, it be- 
comes possible to make selection between the case where 
a synergistic highlighting is to be prevented and the case 
where it is allowed. 

[0126] Needless to say, if the luminance distribution enlarging 
operation and the saturation highlighting operation can 
eventually be suppressed with respect to each other, this 



will do. For example, there may be adopted a method 
wherein when it is necessary to suppress the enlarging 
operation of the contrast enlarging means, the enlarging 
operation of the contrast enlarging means is not sup- 
pressed, but there may be performed a further transfor- 
mation processing which denies the result of enlargement 
in the enlarged image data. It is also possible to obtain 
the same result by darkening the entire image though this 
is different from the enlargement of contrast. This can 
also be said of the saturation highlighting operation. 
[0127] In connection with such a case, it is a further object of the 
present invention to provide a more concrete method for 
suppressing both highlighting operations with respect to 
each other. 

[0128] In the image processing system according to the present 
invention, the highlighting process suppressing means 
sets a correlation so that when one of a parameter which 
represents the degree of luminance distribution enlarge- 
ment in the contrast enlarging means and a parameter 
which represents the degree of saturation highlighting is 
large, the other is small. 

[0129] In the present invention thus constructed, the contrast 
enlarging means transforms image data with use of the 



parameter which represents the degree of luminance dis- 
tribution enlargement, and the saturation highlighting 
means also transforms image data with use of the param- 
eter which represents the degree of saturation highlight- 
ing. Therefore, the correlation of one being large and the 
other small made by the highlighting process suppressing 
means eventually causes the luminance distribution en- 
larging operation and the saturation highlighting opera- 
tion to suppress each other. 

[0130] Thus, the correlation using the parameters representing 
the degree of contrast enlargement and the degree of sat- 
uration highlighting facilitates the processing in the 
present invention. 

[0131] Various methods are available for correlating both param- 
eters. For example, if one parameter is multiplied by a co- 
efficient using the other parameter as denominator, the 
one parameter becomes smaller as the other parameter 
becomes larger. In this case, the other parameter is not 
influenced by the one parameter, but if it is desired that 
both be mutually influenced, this can be done easily by 
another method. Needless to say, there may be adopted a 
method wherein a transformation table to be referred to 
by both parameters and such transformation values as to 



suppress each other are read out. 

[0132] Not only the above methods but also various other meth- 
ods are available for the transformation of image data 
performed by the contrast enlarging means and the satu- 
ration highlighting means. To be concrete, the image data 
transforming operation may be done for each pixel. In the 
case where the image data transformation for contrast 
enlargement and the image data transformation for satu- 
ration highlighting are not performed for each pixel, a 
causal relation between both processing becomes compli- 
cated, and as the case may be it is required to conduct a 
complicated processing for suppressing the two with re- 
spect to each other, or a work area may be needed sepa- 
rately. In contrast therewith, the image data transforming 
operation performed for each pixel is advantageous in 
that the influence of the image data increasing or de- 
creasing process on contrast and saturation becomes 
simple and hence the mutually suppressing process also 
becomes simple. 

[0133] Also in setting the degree of highlighting, one or both of 
the contrast enlarging means and the saturation high- 
lighting means may analyze image data and set the de- 
gree of highlighting. That is, the contrast enlarging means 



and the saturation highlighting means set the degree of 
highlighting automatically and in the course of this au- 
tomation the highlighting process suppressing means 
performs the foregoing suppressing operation. Therefore, 
while the contrast enlarging means sets the degree of 
highlighting, if the degree of highlighting is weakened or 
a change is made from one processing to another by ref- 
erence to, for example, the parameter in the saturation 
highlighting means, such a processing itself leads to con- 
stitution of the highlighting process suppressing means. 
Needless to say, in the case where the degree of high- 
lighting is weakened or a change is made from one pro- 
cessing to another by reference to, for example, the pa- 
rameter in the contrast enlarging means, such a process- 
ing itself leads to constitution of the highlighting process 
suppressing means. 

[0134] It goes without saying that not only forjudging whether 
the image concerned is a natural picture or not on the ba- 
sis of the number of colors but also for judging whether 
the image concerned is to be subjected to image process- 
ing or not, there may be adopted other methods. 

[0135] For example, there may be adopted a method wherein a 
binary image data, such as monochrome image data, is 



determined on the basis of a luminance distribution and, 
if the image data concerned is a binary image data, the 
enlargement of luminance distribution and highlighting of 
saturation may be omitted. 

[0136] As to a binary image, there is no luminance distribution in 
a substantial sense nor is saturation, so once a binary im- 
age data is determined from a luminance distribution, 
there is performed neither enlargement of luminance dis- 
tribution nor highlighting of saturation. Further, since it is 
possible that a binary image data will have a certain color, 
there can be two luminances corresponding respectively 
to the presence and absence of that color. It is possible to 
judge whether a luminance is of that color or not, but 
when there is no suggestive information, it may be judged 
that the image data concerned is a binary black- 
and-white image data when the luminance distribution is 
concentrated on both ends in a reproducible range. That 
is, in the case of a black-and-white image, it can be said 
that the luminance distribution is concentrated on both 
ends in the reproducible range, thus permitting judgment. 

[0^37] It is the formation of a frame that can occur frequently in 
image processing. It is not necessary to apply any image 
processing to the frame. It is therefore a further object of 



the present invention to provide an image processing sys- 
tem, an image processing metliod, and a medium liaving 
an image processing control program recorded thereon, 
capable of performing an image processing most suitable 
to image data having a frame. 

[0138] The image processing system according to the present in- 
vention inputs image data which represents information of 
each of pixels resolved in a dot matrix form from an im- 
age and then executes a predetermined image processing. 
The image processing system is provided with a frame 
discriminating means which, on the basis of the image 
data. Judges a portion including an extremely large num- 
ber of certain pixels to be a frame portion, and an image 
data excluding means which excludes from the image 
processing the image data of pixels having been Judged 
to be a frame portion. 

[0139] In the image processing method according to the present 
invention, a predetermined image processing is per- 
formed for image data which represents information of 
each of pixels resolved in a dot matrix form from an im- 
age. According to the same method, if a portion of the 
image data includes an extremely large number of certain 
pixels, it is Judged to be a frame portion, and the image 



data of the pixels having been judged to be a frame por- 
tion is not subjected to the image processing. 

[0140] Further, the medium according to the present invention 
has an image processing control program recorded 
thereon for inputting image data which represent infor- 
mation of each of the pixels resolved in a dot matrix form 
from an image and performing a predetermined image 
processing, which program comprises the steps of judg- 
ing, on the basis of the image data, a portion to be a 
frame portion if an extremely large number of certain pix- 
els are included therein, and excluding from the image 
processing the image data of the pixels having been 
judged to be a frame portion. 

[0141] In the present invention thus constructed, the frame dis- 
criminating means judges, on the basis of the image data, 
a portion of the image data to be a frame portion if an ex- 
tremely large number of pixels are included therein, and 
the image data excluding means excludes from the image 
processing the image data of the pixels having been 
judged to be a frame portion. 

[0142] That an image possesses a frame occurs frequently. The 
pixels which constitute a frame are almost the same pix- 
els and the number thereof is extremely large as com- 



pared with the number of the other pixels. Therefore, if 
they are included in an extremely large number in a cer- 
tain portion of the image data, this portion is judged to be 
a frame portion and is not subjected to the image pro- 
cessing. 

[0143] The frame which occurs in the image data is black or 

white in many cases. It is therefore a further object of the 
present invention to provide an image processing system 
capable of excluding such a black or white frame effi- 
ciently. In the image processing system according to the 
present invention, the frame discriminating means re- 
gards pixels which take both-end values in an effective 
range of the image data as a candidate for a frame por- 
tion. 

[0144] Needless to say, the color of frame is not limited to black 
and white. It is a further object of the present invention to 
exclude even other frames than black and white frames. 

[0145] In the image processing system according to the present 
invention, the frame determining means totals luminances 
of the pixels into a luminance distribution as a whole 
when the image data is a natural picture and regards a 
prominent luminance portion in the luminance distribu- 
tion as a candidate for a frame portion. 



[0146] If such a prominent luminance portion exists as a mono- 
color frame, only the luminance portion corresponding to 
that color projects. Therefore, a prominent luminance 
portion, if any, is judged to be a frame portion of image 
data, and once a frame portion is detected, the data of the 
frame portion need not be used in the enlargement of lu- 
minance distribution and highlighting of saturation. More 
particularly, if a prominent luminance portion is used as a 
criterion in the luminance enlargement, an effective judg- 
ment may no longer be feasible, so such a portion is 
judged to be a frame portion and is not used in the en- 
largement of luminance distribution. 

[0147] As an example, the luminance portions concentrated on 
both ends in a reproducible range may be judged to be a 
white or black frame portion. A white or black frame is of- 
ten detected and adopted and can also result from trim- 
ming. It corresponds to an end portion in a reproducible 
range. Therefore, the luminance portion concentrated on 
the end portion is judged to be a frame portion. 

[0148] As an example of judging whether the image data is of a 
natural picture or not, there may be used a natural picture 
discriminating means which judges the image data to be 
not of a natural picture when the luminance distribution 



exists in the sliape of a linear spectrum. It can be said that 
a natural picture is characterized by a smooth width of its 
luminance distribution. In most cases, therefore, if the lu- 
minance distribution is in the shape of a linear spectrum, 
one can Judge that the image data is not of a natural pic- 
ture. 

[0149] In some case the image processing system is used alone, 
and in another case it may be incorporated in a certain 
apparatus. Thus, the idea of the invention is not restricted 
to only a limited case, but covers various modes. A 
change may be made as necessary, for example, between 
software and hardware. 

[0150] As an example, the image processing system in question 
may be applied to a printer driver in which inputted image 
data is transformed into image data corresponding to the 
printing ink used and the thus-transformed image data is 
printed using a predetermined color printer. 

[0151] Thus, the printer driver transforms the inputted image 

data correspondingly to the printing ink used, and at this 
time the foregoing processing may be carried out for the 
image data. In the case where a materialized example of 
the idea of the invention takes the form of software in the 
image processing system, such a software is inevitably 



present and utilized also on the recording medium used in 
the system. It is therefore a further object of the present 
invention to provide not only the foregoing image pro- 
cessing system and method but also a medium having an 
image processing control program recorded thereon 
which program substantially controls the image process- 
ing system and method. 
[0152] Needless to say, the recording medium maybe a magnetic 
recording medium or a magneto-optic recording medium, 
or any other recording medium which will be developed in 
the future. Just the same way of thinking is applicable also 
to duplicate products, including primary and secondary 
duplicate products. The present invention is applicable 
also to the case where a communication line is utilized as 
a software supply line. In this case, the software supply 
side using the communication line functions as a software 
supply system and thus the present invention is also uti- 
lized here. 

[0153] Further, even a construction including software as a por- 
tion and hardware as another portion is by no means dif- 
ferent from the concept of the present invention. There 
may be adopted a construction wherein image data are 
partially recorded on a recording medium and are read as 



necessary. It goes without saying tliat tlie invention is ap- 
plicable also to such image processing systems as color 
facsimile machine, color copying machine, color scanner, 

digital still cameral and digital video camera. 
BRIEF DESCRIPTION OF THE DRAWINGS 

[0154] FIG.l is a block diagram showing a concrete hardware 

configuration example of an image processing system ac- 
cording to an embodiment of the present invention; 

[0155] FIG. 2 is a block diagram showing processing performed in 
the interior of a computer; 

[0156] FIG.3 is a schematic block diagram showing a more de- 
tailed configuration of an image processing means; 

[0157] FIG. 4 is a schematic block diagram showing a more de- 
tailed configuration of a printer driver; 

[0158] FIG. 5 is a schematic block diagram showing another ap- 
plication example of the image processing system; 

[0159] FIG. 6 is a schematic block diagram showing a further ap- 
plication example of the image processing system; 

[0160] FIG. 7 is a flowchart showing an image processing in an 
application according to the present invention; 

[0161] FIG. 8 is a diagram showing an image before transforma- 
tion; 

[0162] FIG. 9 is a diagram showing a sampling cycle; 



[0163] FIG. 10 is a diagram showing the number of sampling pix- 
els; 

[0164] FICS.11(A)-(C) are a diagrams showing a relation between 
an image before transformation and a sampling image; 

[0165] FIG. 12 is a flowchart showing a luminance distribution 

sampling and processing portion in the image processing 
system; 

[0166] FIG. 13 is a flowchart showing a luminance transformation 

processing portion in the image processing system; 
[0167] FIG. 14 is a diagram showing a black-and-white image; 

[0168] FIG. 15 is a diagram showing a luminance distribution of 

the black-and-white image; 
[0169] FIG. 16 is a diagram showing an image having a frame 

portion; 

[0170] FIG. 17 is a diagram showing a luminance distribution of 
the framed image; 

[0171] FIG. 18 is a diagram showing processing for end portions 
of a luminance distribution; 

[0172] FIG. 19 is a diagram showing enlargement of a luminance 
distribution and a reproducible luminance range; 

[0173] FIG. 20 is a diagram showing limitation of a luminance dis- 
tribution enlarging ratio; 

[0174] FIG. 21 is a diagram showing a transformation table for 



use in enlarging a luminance distribution; 
[0175] FIG. 22 is a diagram showing a brightening concept 
byycorrection; 

[0176] FIG. 23 is a diagram showing a darkening concept 
byycorrection; 

[0177] FIG. 24 is a diagram showing a correlation in changing lu- 
minance byycorrection; 

[0178] FIG. 2 5 is a flowchart showing a saturation transforming 
process performed in the image processing system; 

[0179] FIG. 26 is a schematic diagram showing a totaled state in a 
saturation distribution; 

[0180] FIG. 27 is a diagram showing a saturation distribution of 
image data which is not a natural picture; 

[0181] FIG. 28 is a diagram showing a saturation distribution of 
an image having a frame portion; 

[0182] FIG. 29 is a diagram showing a relation between saturation 
A and a saturation highlighting index S; 

[0183] FIG. 30 is a diagram showing a saturation highlighting 
limit in Luv space; 

[0184] FIG. 31 is a flowchart showing image processing according 
to the present invention; 

[0185] FIG. 32 is a flowchart showing an outline of a contrast en- 
larging process; 



[0186] FIG. 33 is a flowchart showing an outline of a saturation 
highlighting process; 

[0187] FIG. 34 is a diagram showing an application concept of a 
highlighting suppressing process applied in the embodi- 
ment; 

[0188] FIG. 35 is a diagram showing an application concept of a 
highlighting suppressing process applied in a modified 
embodiment according to the present invention; 

[0189] FIG. 36 is a diagram showing an application concept of a 
highlighting suppressing process applied in another mod- 
ified embodiment according to the present invention; 

[0190] FIGS.37 (A) and (B) are diagrams showing an application 

concept of a highlighting suppressing concept applied us- 
ing a transformation table in a further modified embodi- 
ment according to the invention; 

[0191] FIGS.38 (A) and (B) are diagrams showing an application 
concept of a highlighting suppressing process using a 
non-interference area; 

[0192] FIG. 39 is a flowchart showing an edge highlighting pro- 
cess provided as an image processing; 

[0193] FIG. 40 is a diagram showing an unsharp mask; 

[0194] FIG. 41 is a diagram showing an improved unsharp mask; 



[0195] FIG. 42 is a diagram showing directions for comparison 

with adjacent pixels in image data; 
[0196] FIG. 43 is a flowchart showing in what procedure a printer 

driver performs a printing process; 
[0197] FIG. 44 is a flowchart of a color transformation processing 

using a cache + interpolating calculation process; 
[0198] FIGS.45(A)-(C) are diagrams showing a concept of 8-point 

interpolating calculation; 
[0199] FIG. 46 is a diagram showing the contents of a cache table; 

[0200] FIG. 47 is a flowchart of a pre-gray level transforming pro- 
cess; and 

[0201] FIG. 48 is a diagram showing in what manner an error is 

diffused in each pixel. 
DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0202] Embodiments of the present invention will be described 
hereinunder with reference to the accompanying draw- 
ings. 

[0203] FIG.l shows as a block diagram a concrete hardware con- 
figuration example of an image processing system ac- 
cording to an embodiment of the present invention. 

[0204] In the figure, a scanner 11, a digital still camera 12 and a 
video camera 14 are provided as image input devices 10. 



A computer body 21 and a hard disk 22 are provided as 
image processors 20 wliicli play main roles in image pro- 
cessing including image discrimination. Further, a printer 
31 and a display 32 are provided as image output devices 
30 for the display and output of images after image pro- 
cessing. 

[0205] In the interior of the computer body 21 there is performed 
processing as shown in FIG. 2. As shown in the same fig- 
ure, an operating system 21a operates in the interior of 
the computer body 21, and a printer driver 21b and a 
video driver 21c are incorporated therein correspondingly 
to the printer 31 and the display 32, respectively. On the 
other hand, an application 2 Id is controlled in its execu- 
tion of processing by the operating system 21a, and 
where required, it cooperates with the printer driver 21b 
and the video driver 21c to execute a predetermined im- 
age processing. 

[0206] In this embodiment, the scanner 11 and the digital still 
camera 12 as image input devices 10 output gradation 
data of RGB (red, green, blue)as image data. As the image 
output devices 30, the printer 31 requires CMY (cyan, ma- 
genta, yellow) or CMYK (CMY + black) as an input of bi- 
nary colorimetric data, and the display 32 requires grada- 



tion data of RGB as an input. Therefore, a concrete role of 
the computer body 21 as an image processor 20 is to in- 
put gradation data of RGB, prepare gradation data of RGB 
having been subjected to a necessary highlighting pro- 
cess, and cause it to be displayed on the display 32 
through the video driver 21c or cause it to be printed by 
the printer 31 after transformation into binary data of 
CMY through the printer driver 21b. 

[0207] According to the image processing performed in this em- 
bodiment, upon input of image data into the computer 
body 21, a number-of-color detecting means 21dl in the 
application 2 Id detects the number of colors used in the 
inputted image data, then an image discriminating means 
21d2 judges the type of image, and an image processing 
means 21d3 performs an appropriate image processing 
automatically which processing is set beforehand accord- 
ing to the type of image. The image data after the image 
processing is displayed on the display 32 through the 
video driver 21c and, after confirmation, are transformed 
into printing data by the printer driver 21b, which is 
printed by the printer 31. 

[0208] In this embodiment, for the image processing means 
21d3 which is set according to the type of image, there 



are provided a contrast enlarging process, a saturation 
liigliligliting process and an edge liighlighting process, as 
shown in FIG. 3. In the printer driver 21b which transforms 
image data into printing data, as shown in FIG. 4, there are 
provided a rasterizer 21bl for rasterizing the printing 
data, a color transforming section 21b2 which performs 
color transformation by pre-gray level transformation or 
by a combination of cache and interpolating calculation, 
and a gray level transforming section21b3 which performs 
a binary coding process for gradation data after the color 
transformation. 

[0209] Although in this embodiment a computer system is inter- 
posed between the image input and output devices to 
perform the image processing, such a computer system is 
not always needed. There may be adopted such a system 
configuration as shown in FIG. 5 wherein image processors 
are mounted for highlighting contrast within the digital 
still cameral2a, and image data after transformation is 
displayed on a display 32a or printed by a printer 31a. As 
shown in FIG. 6, in the case of a printer 31b which inputs 
and prints image data without going through a computer 
system, it is possible to construct the printer so that im- 
age data inputted through a scanner lib, a digital still 



camera 12b, or a modem 13b, is highlighted in its con- 
trast automatically. 

[0210] FIG. 7 is a flowchart corresponding to the image process- 
ing application. In steps S102 and S104, luminances are 
totaled into a luminance distribution and the number of 
colors used is detected. 

[0211] Reference will first be made below to the luminance distri- 
bution detecting process. 

[0212] Before explaining in what manner luminance is to be rep- 
resented, description will be directed to pixels to be to- 
taled into a luminance distribution. As in step S102 shown 
in FIG. 7, there is performed processing for thinning out 
pixels concerned. As shown in FIG. 8, if the inputted image 
is a bit map image, it is formed in a two-dimensional dot 
matrix shape comprising a predetermined number of dots 
in the longitudinal direction and a predetermined number 
of dots in the lateral direction. For obtaining an accurate 
luminance distribution it is necessary to check luminance 
with respect to all the pixels. However, the distribution 
detecting process in question aims at detecting the num- 
ber of colors used indirectly from luminance, so it is not 
always required to be accurate. Therefore, it is possible to 
effect thinning-out of pixels within a certain range of er- 



ror. According to a statistical error, an error relative to tlie 
number of samples N is expressed generally as 
l/(N**(l/2)) where ** stands for a power. Accordingly, for 
carrying out the processing with an error of 1% or so, N is 
equal to 10000. 

[0213] In this connection, the bit map screen shown in FIG. 8 in- 
cludes (width) x(height) number of pixels, and a sampling 
cycle ratio is assumed to be as follows: 

[0214] Ratio = min (width, height)/A + 1... (8) 

[0215] In the above expression, min(width, height) indicates the 
smaller one of width and height, and A is a constant. Fur- 
ther, the sampling cycle ratio indicates the number of 
pixel at which sampling is performed. Then-marked pix- 
els in FIG. 9 represents a sampling cycle ratio = 2. That is, 
sampling of one pixel is performed at every two pixels in 
the longitudinal and lateral directions. Thus, sampling is 
performed at every other pixel. Given that A is equal to 
200, the number of sampling pixels in one line is as 
shown in FIG. 10. 

[0216] As is seen from the same figure, the number of samples is 
at least 100 pixels or more when there is a width of 200 
pixels or more, except the sampling cycle ratio = 1 at 
which sampling is not conducted. Therefore, in the case of 



200 pixels or more in the longitudinal and lateral direc- 
tions, there is ensured(100 pixels)x(100 pixels) = (10000 
pixels) and error can be kept to 1% or less. 

[0217] jhe sampling cycle ratio is based on min(width, height) is 
for the following reason. If width>> height as in the bit 
map image shown in FIG. 11(a) if the sampling cycle ratio 
is determined on the basis of the longer width, there may 
occur a phenomenon such that pixels are sampled longi- 
tudinally in only two upper- and lower-end lines as in 
FIG. 11(b). However, if the sampling cycle ratio is deter- 
mined on the basis of the smaller one of width and height 
becomes possible to effect thinning-out of pixels so as to 
include an intermediate portion also in the longitudinal 
direction of the smaller one as in FIG. 11(c). 

[0218] In this example the thinning-out of pixels is performed at 
an accurate sampling cycle for pixels in both longitudinal 
and lateral directions. This is suitable for the case where 
successively inputted pixels are processed while they are 
thinned out. However, in the case where all the pixels 
have been inputted, coordinates may be designated ran- 
domly in the longitudinal or lateral direction to select pix- 
els. By so doing, when a minimum required number of 
pixels, say 10000 pixels, have been determined, the sam- 



pling process may be repeated randomly up to 10000 
pixels and be stopped when the 10000 pixels have been 
sampled. 

[0219] If the image data of the thus-selected pixels has lumi- 
nances as constituent elements, a luminance distribution 
can be obtained using the luminance values. On the other 
hand, even in the case of image data wherein luminance 
values are not direct component values, it is indirectly 
provided with component values indicative of luminance. 
Therefore, if transformation is made from a color specifi- 
cation space wherein luminance values are not direct 
component values to a color specification space wherein 
luminance values are direct component values, it is possi- 
ble to obtain luminance values. 

[0220] It is not that the color transformation between different 
color specification spaces is determined in a unitary man- 
ner, but it is required to first determine a correlation of 
color spaces including component values as coordinates 
and then perform transformation successively while mak- 
ing reference to a color transformation table with the cor- 
relation stored therein. Because of the table, component 
values are represented as gray scale values, and in the 
case of 256 gray scales having three-dimensional coordi- 



nate axes, the color transformation table must cover 
about 16,700,000 (256x256x256) elements. In view of an 
effective utilization of the memory resource, correlations 
are prepared usually with respect to sporadic lattice 
points, not all the coordinate values, and an interpolating 
calculation is utilized. Since this interpolating calculation 
is feasible through several multiplications and additions, 
the calculation volume becomes vast. 

[0221] More particularly, if a color transformation table of full 

size is used, the processing volume becomes smaller, but 
the table size poses an unreal problem, and if the table 
size is set at a real size, the calculation volume becomes 
unreal in many cases. 

[0222] In this embodiment, in view of such circumstances, there 
is adopted the following transforming expression which 
determines luminance from the three primary colors of 
RGB. More specifically, as to the luminance yp at point P, it 
is set as follows on the basis of component values 
(Rp,Gp,Bp) of RGB: 

[0223] yp = o.30Rp + 0.59Gp + O.llBp... (9) 

[0224] Using this expression, a luminance value can be obtained 

by only three multiplications and two additions. 
[0225] In this embodiment there is adopted such a transforming 



expression because the color specification space of RGB is 
used, but at the bacl<ground thereof there exists a nature 
such that each component value represents the brightness 
of color and therefore, when viewed each independently, 
corresponds to linearly to luminance. More roughly 
speaking, therefore, it is not impossible to simplify as fol- 
lows without considering each addition ratio: 
[0226] yp = (Rp + Cp + Bp)/3... (10) 

[0227] After luminances have been totaled by such a thinning- 
out process, the number of luminance values not '0' in the 
number of pixels is counted in step S104, whereby the 
number of colors used can be detected. Although in step 
S102 not only luminance values but also saturation values 
are totaled to detect a distribution thereof, this totaling 
process will be described later. 

[0228] Once the number of colors used is detected in this way, a 
comparison is made in step S106 between the detected 
number of colors and a predetermined threshold value, 
and if the number of colors used is larger, it is judged that 
the image data in question is of a natural picture. For ex- 
ample, the threshold value may be set at '50' colors. As 
other examples of image data other than natural pictures 
— , — there are business graphs and drawing type im- 



ages, which are read using the scanner 11. In some cases, 
a computer graphic image is read through the scanner 11 
or is input from a networl< through the modem 13b. 

[0229] In the case of a natural picture, a related flag is set in step 
S108. The reason why the flag is set is that it is intended 
to transmit the result of judgment also to the printer 
driver 21b in an operation other than the image process- 
ing in the application 21d. Although in this embodiment 
there is provided only one threshold value to Judge 
whether the image data concerned is of a natural picture 
or not, there may be performed a more detailed judgment 
in the range corresponding to the number of colors used. 
For example, even in the case of a computer graphic im- 
age, the number of colors sometimes becomes larger de- 
pending on gradation or may become larger due to a 
blunt edge portion at the time of reading by the scanner 
11 despite the actual number of colors being not so large. 
In this case there may be adopted a method wherein such 
image data are classified between natural and unnatural 
pictures and there is conducted only an edge highlighting 
process even without conducting the image processing for 
natural pictures as will be described later. 

[0230] As to a natural picture, contrast is enlarged in step SI 10, 



saturation is liigliliglited in step S112, and edge is liigli- 
lighted in step S114. 

[0231] The contrast enlarging process is shown in FIGS. 12 and 13 
in terms of flowcharts. As shown in the same figures, the 
following points should be taken into consideration before 
obtaining a luminance distribution of the pixels selected 
by the above thinning-out processing. 

[0232] One point relates to the case where the image concerned 
is a binary image such as a black-and-white image. In the 
case of a binary image, including a black-and-white im- 
age, the concept of highlighting contrast is inappropriate. 
In such a black-and-white image as shown in FIG. 14, a 
luminance distribution of this image concentrates on both 
ends in a reproducible range, as shown in FIG. 15. Basi- 
cally, it concentrates on the gray scales '0' and "255." 

[0233] Therefore, the judgment in step S124 as to whether the 
inputted image is a black-and-white image or not can be 
done on the basis of whether the sum of pixels at gray 
scales of '0' and '255' is coincident or not with the number 
of the pixels thinned out and selected. If the image is a 
black-and-white image, the processing flow shifts to step 
S126 in which an unenlarging process is executed to stop 
the processing without going through the processing de- 



scribed below. In this embodiment, the distribution sam- 
pling process and the luminance transforming process are 
broadly classified, and in this unenlarging process a flag 
is raised so as not to execute the latter stage of luminance 
transforming process, thereby bringing this distribution 
sampling process to an end. 

[0234] Binary data is not limited to black and white, but there can 
be even colored binary data. Also in this case it is not 
necessary to carry out the contrast highlighting process. If 
a check of the state of distribution shows that distribution 
is concentrated on only two values (one is usually '0'), the 
data concerned is regarded as a binary data and the pro- 
cessing may be interrupted. 

[0235] The second point to be taken into account is whether the 
inputted image is a natural picture or not. This Judgment 
can be done on the basis of the flag set in step S108. If 
the number of colors is small, the luminance distribution 
is sparse, and in the case of a business graph or the like, 
it is represented in the form of a linear spectrum. Whether 
the distribution is in the form of a linear spectrum or not 
can be judged in terms of an adjacent ratio of luminance 
values which are not '0' in the number thereof distributed. 
To be more specific, a judgment is made as to whether 



there is a distribution number in adjacent luminance val- 
ues which are not '0' in the number thereof distributed. In 
terms of the distribution number, if at least one of two 
adjacent luminance values is in an adjacent state, there is 
done nothing, while if neither is in an adjacent state, 
counting is performed. As a result. Judgment may be 
made on the basis of the ratio between the number of lu- 
minance values which are not '0' and the counted value. 
For example, if the number of luminance values not '0' is 
50 and that of those not in an adjacent state is 50, it is 
seen that the luminance distribution is in the form of a 
linear spectrum. 
[0236] In the case where the image processing program is exe- 
cuted through the operating system, the Judgment as to 
whether the inputted image is a natural picture or not can 
be made using an image file extender. In the bit map file, 
particularly for photographic images, the file is com- 
pressed and there often is used an implicit extender for 
representing the compressing method. For example, the 
extender 'JPG' shows that the compression is made in JPEG 
format. Since the operating system manages the file 
name, if an inquiry is issued from the printer driver side 
for example to the operating system, the extender of the 



file is replied, so there may be adopted a procedure of 
Judging tlie image concerned to be a natural picture on 
the basis of that extender and then highlighting the con- 
trast. In the case of an extender peculiar to a business 
graph such as 'XLS,' it is possible to determine that con- 
trast highlighting is not performed. 

[0237] jhe third point to be taken into account is whether the 
marginal portion of the image is framed as in FIG. 16. If 
the frame portion is white or black, it influences the lumi- 
nance distribution and a linear spectral shape appears at 
each end in a reproducible range as in FIG. 17. At the same 
time, a smooth luminance distribution also appears inside 
both ends correspondingly to the interior natural picture. 

[0238] Needless to say, since it is more appropriate not to con- 
sider the frame portion in the luminance distribution, a 
Judgment is made as to whether the sum of pixels at the 
gray scales '0' and '255' is sufficiently large and coincides 
with the number of pixels which have been thinned out 
and selected. If the answer is affirmative, it is Judged that 
there is a frame portion, and in step S132 there is per- 
formed a frame processing. In this frame processing, the 
number of pixels in the gray scales of '0' and '255' in the 
luminance distribution is set to '0' in order to ignore the 



frame portion. By so doing, the processing wliich follows 
can be done on the assumption that there is no frame 
portion. 

[0239] Although the frame portion being considered in this ex- 
ample is white or black, there sometimes is a frame of a 
specific color. In this case, there appears a linear spectral 
shape projecting from the original smooth curve of the lu- 
minance distribution. Therefore, as to a linear spectral 
shape which causes a great difference between adjacent 
luminance values, it is considered to be a frame portion 
and not considered in the luminance distribution. Since it 
is possible that the said color may be used in the other 
portion than the frame portion, there may be allocated a 
mean of both adjacent luminance values. 

[0240] After the above considerations, if the luminance distribu- 
tion is to be enlarged, there are obtained both ends of the 
distribution in step S136. 

[0241] In the case of a natural picture, its luminance distribution 
appears generally in a chevron shape, as shown in FIG. 18, 
provided various positions and shapes can appear. The 
width of the luminance distribution depends on the posi- 
tions of both ends of the distribution. However, mere ex- 
tended skirt positions corresponding to the distribution 



number of '0' cannot be set at both ends. This is because 
in the sicirt portions the distribution number shifts in the 
vicinity of '0' and when viewed statistically there occurs an 
unlimited change which approaches '0.' 

[0242] For this reason, portions located respectively inside the 
highest luminance side and the lowest luminance side in 
the distribution range by a certain distribution ratio are 
regarded as both ends. In this embodiment the said dis- 
tribution ratio is set at 0.5%, as shown in FIG. 18. Needless 
to say, this ratio may be changed as necessary. By thus 
cutting off both upper and lower ends by only a certain 
ratio it becomes possible to ignore white or black spots 
caused by noise for example. Without this processing, 
even one white or black spot would form both ends in the 
luminance distribution, so that in many cases the bottom 
and top ends correspond to '0' and '255' in gray scale. But 
this does not occur if the portion located inside the top 
end by 0.5% pixels is made an end portion. 

[0243] In the actual processing, there is calculated 0.5% of the 

pixel to be processed (the total number of pixels selected 
by thinning-out or the total number of pixels obtained by 
removing the pixels corresponding to the frame portion), 
then the respective distribution numbers are totaled sue- 



cessively toward the inside from tlie upper- and lower- 
end luminance values in the reproducible luminance dis- 
tribution, and there is determined a luminance value cor- 
responding to 0.5%. These upper end side and lower end 
side will hereinafter be referred to as y and y , re- 

max min 

spectively. 

[0244] Although in this embodiment both upper and lower ends 
are determined through such a processing for the lumi- 
nance distribution, it is also possible to determine both 
ends by a statistical processing. For example, there may 
be adopted a method wherein a portion of a certain per- 
cent relative to a mean value of luminance values is re- 
garded as an end portion. 

[0245] The above processing corresponds to the distribution de- 
tecting processing. Next, a description will be given of a 
luminance transforming process which performs the 
transformation of image data on the basis of the lumi- 
nance values y and y thus obtained. If an unenlarging 

max min 

process is executed in step S126, this is detected by ref- 
erence to a predetermined flag in step S142 and the im- 
age processing is ended without performing the process- 
ings which follow. A basic transformation of luminance is 
performed in the following manner. Given that the repro- 



ducible luminance range is 0 to 255, tlie luminance Y after 
transformation is determined from the luminance y before 
transformation and the maximum value y and minimum 

max 

value y in the luminance distribution range in accor- 

min 

dance with the following expression: 
[0246] Y = ay + b...(2) 

[0247] where, a = 255D(y -y ) ...(3)' 

max min 

[0248] b = -aDy or 255-aDy ...(4) 

mi max 

[0249] In the above transforming expression, if Y < 0, Y=0, and 
if Y>255, Y=255. In the same expression it can be said 
that a stands for inclination and b offset. According to this 
transforming expression, a luminance distribution having 
a certain narrow width can be widened up to a repro- 
ducible range, as shown in FIG. 19. Basically in the en- 
largement of a luminance distribution range, the original 
and enlarged ranges coincide with each other in the area 
of histogram because the number of pixels does not 
change. 

[0250] When a luminance distribution is enlarged by making the 
most of the reproducible range, there sometimes occurs a 
bald white impression of a highlight portion or a solid 
black impression of a high-shadow portion. To avoid this. 



the reproducible range is limited in this embodiment. 
More particularly, there remains an unenlarged area of 5 
in terms of a luminance value from each of the upper and 
lower ends in the reproducible range. As a result, the pa- 
rameters in the transforming expression become as fol- 
lows: 

[0251] a = 245/(y -y )...(11) 

max min 

[0252] b = 5-aDy or 250-aDy ...(12) 

min max 

[0253] In this case, transformation is not performed in the ranges 
of y < y and y 

min max 

[0254] Thus, the area of 5 in terms of a luminance value from 

each end portion is set as an unenlarged area for retaining 
both highlight portion and high-shadow portion. How- 
ever, if an image output device used permits both high- 
light portion and high-shadow portion to be reproduced 
relatively easily, the above area may be narrowed, or in 
the case where the reproducing power is weaker, the 
above area may be widened. It is not always required to 
stop the enlargement completely. The enlargement ratio 
may be gradually limited at the border region. 

[0255] FIG. 20(a) shows a narrow luminance distribution of an im- 
age. If the enlargement ratio of luminance distribution 



(corresponding to a) is applied in the manner described 
above, there sometimes is obtained a very large enlarge- 
ment ratio in proportion to a reproducible range. In this 
case, although it is natural that the contrast width form 
the brightest portion to the darkest portion should be 
narrow in a dusk state, for example at nightfall, an at- 
tempt to enlarge the image contrast to a great extent may 
result in transformation into an image as if the image 
were a daytime image. Since such a transformation is not 
desired, a limit is placed on the enlargement ratio so that 
the parameter a does not exceed 1.5 (□2). By so doing, a 
dusk state is expressed as such. 
[0256] When no limitation is placed on the enlargement ratio, the 
result is as indicated with a dot-dash line in FIG. 20(a). As 
shown in the same figure, there remains no extra portion 
in the reproducible range after transformation. However, 
in the case where the enlargement range is restricted, as 
shown by two-dots-dash line in Fig. 20(b), there occurs 
freedom as to where the distribution after transformation 
should be. In some case the whole may be too bright or 
too dark. 

[0257] In this embodiment, therefore, transformation is made in 
such a manner that the ratio of areas (ml:m2) remaining 



on the upper end side and the lower end side in the re- 
producible range of a luminance distribution before trans- 
formation coincides with the ratio of areas 
(nl:n2)remaining on the upper and lower end sides after 
transformation. In this case, the parameter b is deter- 
mined in the following manner. 

[0258] In the luminance distribution of image before transforma- 
tion, 

[0259] m 1 = y 

min 

[0260] rn2 = 255-y 

max 

[0261] Now, ml + m2 = 255-( y -y ). Therefore, if y =y 

max min dif 

-y . , 

max mm 

[0262] ml+m2 = 255-y 

dif 

[0263] In the luminance distribution of image after transforma- 
tion, 

[0264] nl=Y 

min 

[0265] n2 = 255-Y Likewise, 

max 

[0266] nl + n2 = 255-(Y -Y ) 

max min 

[0267] = 255-a(y -y ) 

max min 

[0268] = 255-aDy . 



[0269] Since ml:m2 = nl:n2, 
[0270] nl = ml(nl+n2)/(ml+m2) 
[0271] =y (255-aDy J/(255-y J 

mm dif dif 

[0272] On the other hand, since Y = ay + b, b = Y - ay, and 
hence 

[0273] b = Y -any 

min min 

[0274] =Y . {(255-aay^,J/(255-y^>a}...(13) 

mm dif dif 

[0275] Further, the parameter b can be obtained using Y as 

max 

follows: 
[0276] b = Y -any 

max max 

[0277] = 255-(255-y )(255-aDy J/(255-y J-aDy ...(14) 

max dif dif max 

[0278] Now, Step S144 ends. 

[0279] If the above transforming expression (Y=ay+b) is carried 
out at every transformation of luminance, this is illogical. 
This is because the range of luminance y can only be 0 to 
255 and hence the luminance Y after transformation can 
be obtained beforehand for all of the values which the lu- 
minance y can take. Therefore, this correlation is deter- 
mined in step S146 and is stored as such a table as shown 
in FIG. 21. 



[0280] At the time when such a transformation table has been 

formed, it becomes possible to change image data. How- 
ever, not only highlighting the contrast by such enlarge- 
ment of the luminance range but also adjustment of 
brightness is also very effective in attaining the intended 
object. For example, in the case where the expansion of a 
luminance distribution is on a dark side as a whole, as in- 
dicated with a solid line in FIG. 22, the expansion is 
preferably shifted to a bright side as a whole, as indicated 
with a chain line in the same figure. Conversely, when the 
expansion of a luminance distribution is on the bright 
side as a whole, as indicated with a solid line in FIG. 23, 
the expansion is preferably shifted to the dark side as a 
whole, as indicated with a chain line in the same figure. 

[0281] In this embodiment, as a result of various experiments, 

there is adopted a method wherein a median y in a lu- 

minance distribution is determined and when it is smaller 

than 85, it is judged that the image concerned is a dark 

image, and the image is brightened by makingycorrection 

which corresponds to the followingyvalue: 

[0282] Y=y /85...(15)or 
med 

[0283] Y=(y /85)**(l/2) ...(16) 
med 



[0284] In this case, even whenyis smaller thanO.Z.yis assumed 
equal to 0.7. This is because, without such a limitation, a 
night image would become an image like a daytime im- 
age. Excessive brightness would result in a whitish image 
of weak contrast as a whole, so it is desirable to highlight 
saturation at the same time. 

[0285] On the other hand, in the case a median y larger than 

med 

128, it is Judged that the image concerned is a bright im- 
age, and the image is darkened byycorrection which cor- 
responds to the followingyvalue: 

[0286] Y=y / 128 or.. .(17) 
med 

[0287] Y=(y /128)**(l/2) ...(18) 
med 

[0288] In this case, even whenyis larger than 1.3,yis assumed 
equal to 1.3 and thus a limit is set to prevent excessive 
darkness. Excessive darkness would result in the image 
being colored too much into a dark image, so it is desir- 
able to weaken the degree of saturation highlighting. For 
an object in a bright background, however, such a dark- 
ening process sometimes exert a bad influence on the im- 
age. This is because, for example, in a landscape image 
wherein the sky occupies a half of the image or in a sou- 
venir picture on a fine day the human face is easily 



crushed dark due to counter light. Since both darl< and 
bright portions are mixed together in such images, a 
standard deviation ystd of luminance is relatively high in 
many cases. Therefore, when the standard deviation ystd 
is larger than 70, theycorrection for darkening is not per- 
formed. 

[0289] It is optional whether theycorrection is to be made for the 
luminance distribution before transformation or for the 
luminance distribution after transformation. FIG. 24 shows 
a correlation in the case ofycorrection being made. Ify< 1, 
there is obtained an upwardly expanded curve, and ify>l, 
there is obtained a downwardly expanded curve. The re- 
sult of theycorrection may be reflected in the table shown 
in FIG. 22. Theycorrection is made for the table data in 
step S148. Lastly, in step S510 there is made transforma- 
tion of image data. The correlations so far obtained are 
for the transformation of luminance and are not trans- 
forming relations for the component values (Rp,Gp,Bp) on 
RGB coordinate axes. However, the transforming expres- 
sion (2) is also applicable to the correlation with the com- 
ponent values (Rp,Gp,Bp) of RGB. That is, the component 
values (R,G,B) are also obtainable as follows relative to the 
component values(rr, gg, bb) before transformation: 



[0290] R = aDrr+b...(19) 
[0291] G = aDgg+b...(20) 
[0292] B = aDbb+b...(21) 

[0293] This is apparent from the fact that both expressions (2) 

and (4) represent linear correlations. Further, correspond- 
ingly to the range of 0 to 255 in gray scale of the lumi- 
nances y and Y, the component values of RGB, i.e., 
(r,g,b),(R,G,B), are also in the same range. It follows that 
the foregoing transformation tables of the luminances y 
and Y can be utilized as they are. 

[0294] Thus, in step S150, the processing of making reference to 
the transformation tables corresponding to the expres- 
sions (19)0(21) to obtain image data (R,G,B) after trans- 
formation is repeated with respect to the image data 
(rr,gg,bb) of all the pixels. 

[0295] Tiie entire operation flow is summarized as follows. 

[0296] In step S102, after the luminance y is obtained with re- 
spect to each of the thinned-out pixels to determine a lu- 
minance distribution, a judgment is made in step S124 as 
to whether the image concerned is such a binary image as 
black-and-white image or not, while in step S128 it is 
judged whether the image is a natural picture or not. In 



step S130, it is judged wliether a frame portion is in- 
cluded in the image data or not, except tlie case wliere 
the image is a binary image or is not a natural picture, 
then the frame portion if any is removed and the areas of 
0.5% from the upper and lower ends of the luminance dis- 
tribution obtained are removed to obtain both ends y 
and y of the distribution. 

max min 

[0297] After both ends y and y of the luminance distribution 

max min 

have been obtained, parameters a and b are determined 
from the following relationships in step S144 and a trans- 
forming relation from luminance y to luminance Y is 
stored in a table in step 146: 
[0298] Y = ay + b ...(2) 

[0299] a = 245/(y -y )...(!!) 

max min 

[0300] b is any of the following: 

[0301] b = 5-aDy or 250-aDy ...(12) 

min max 

[0302] b = Y . {(255-any_^J/(255-y^J-a}...(13) 

mm dif dif 

[0303] b = 255-(255-y )(255-aDy )/(255 -y >aDy ...(14) 

max dif dif max 

[0304] In step S148,YCorrection is performed as necessary and in 
step S150 there is made transformation of image data 
with respect to all the pixels. 



[0305] Needless to say, as mentioned above, such an image pro- 
cessing is not performed in tlie case wliere tlie image 
concerned is a binary image or is not a natural picture. 
However, when the image processing according to the 
present invention is performed, it becomes possible to 
obtain a vivid image sharp in light and shape by making 
correction so as to widen the luminance range despite a 
very weak contrast in the state of the photograph. 

[0306] Although in the above example the enlargement ratio is 
limited to a predetermined certain value, a modification 
may be made so that it can be selected by the user 
through a predetermined GUI on the computer 21. It is 
also possible to let the user designate part of the image 
data and execute such a contrast highlighting process 
only within the range concerned. 

[0307] Turning back to the flowchart shown in FIG. 7, the en- 
largement of contrast is followed by saturation highlight- 
ing in step S112. This saturation highlighting process is 
shown in FIG. 25. Also in the saturation highlighting pro- 
cess, a saturation distribution is obtained beforehand 
from image data and then a saturation highlighting coeffi- 
cient S is determined from the saturation distribution. In 
this case, as is the case with the luminance distribution, it 



is not necessary to total saturation values of all pixels. 
Saturation values are obtained and totaled with respect to 
the pixels thinned out in step S102. If image data has sat- 
uration values as its component factors, it is possible to 
determine a saturation distribution by using the saturation 
values. Even image data with saturations being not direct 
component factors are indirectly provided with component 
values representative of saturation. Therefore, saturation 
values can be obtained by performing transformation 
from a color specification space wherein saturations are 
not direct component factors into a like space wherein 
saturation values are direct component values. For exam- 
ple, in Luv space as a standard colorimetric space, the L 
axis represents luminance (lightness), while U and V axes 
represent hue. In this case, the distance from an inter- 
secting point from both U and V axes represents satura- 
tion, so saturation is substantially represented by 
(U**2+V**2)**(l/2). However, the calculation volume re- 
quired for such transformation is vast. 

[0308] In this embodiment, therefore, a saturation value X is de- 
termined as follows by directly utilizing standard grada- 
tion data of RGB as image data: 

[0309] X = |G + B-2xR|...(5) 



[0310] Saturation usually takes a value of 0 in the case of R=G=B 
and takes a maximum value in the case of a monocolor of 
RGB or a mixture of any two colors thereof at a predeter- 
mined ratio. From this nature it is possible to express sat- 
uration appropriately, but also in accordance with the ex- 
pression (5) there is obtained a maximum value of satura- 
tion in the case of a monocolor of red or light blue which 
is a mixed color of green and blue, with saturation being 
0 when the components are uniform. Also as to monocol- 
ors of green and blue, about half of the maximum value is 
obtained. 

[0311] Where components are represented by almost equal hue 
component values as in the color specification space of 
RGB in which the components represent colors each inde- 
pendently, it is possible to make substitution by the fol- 
lowing expressions: 

[0312] x'= |R+B-2xG|...(6) 

[0313] X' = |G + R-2xB|...(7) 

[0314] However, the foregoing expression (5) afforded the best 
result. 

[0315] If saturations of the pixels after thinning-out are totaled 
into a saturation distribution from the image data of RGB 



and in accordance with tlie expression (5), it is seen that 
the saturations are distributed in the range from a mini- 
mum value of 0 to a maximum value of 511 and that the 
distribution is substantially like that shown in FIG. 26. 
[0316] However, also in the saturation highlighting process it is 
necessary that binary black-and-white images, unnatural 
images such as drawing type images, and framed images, 
be excluded from the processing. In the case of a binary 
black-and-white and image, saturation values X concen- 
trate on 0, while in the case of a drawing type business 
graph there is obtained such a spectral shape as shown in 
FIG. 27. As to a framed image, as shown in FIG. 28, there is 
a tendency such that the saturation values X concentrate 
on 0 despite a gentle distribution. Therefore, these im- 
ages are identified in steps S204 to S210 and are ex- 
cluded, then the degree of saturation highlighting is de- 
cided. 

[0317] More specifically, on the basis of a saturation distribution 
obtained by totaling of saturation values, a saturation 
highlighting index of the image concerned is determined 
in step S212. If the saturation distribution obtained is as 
shown in FIG. 26, then in this embodiment a range occu- 
pied by the higher 16% in terms of a distribution number 



is determined within an effective range of pixels. Tlien, on 
the basis of the assumption that the lowest saturation A in 
the said 16% range represents the saturation of the image, 
a saturation highlighting index S is determined using the 
following expressions: 
[0318] |fA<92, 

[0319] s = -Ax(10/92)+50...(22) 

[0320] If 92^A < 184, 

[0321] s = -Ax(10/46)+60...(23) 

[0322] If 184^A < 230, 

[0323] S = -Ax(10/23)+100...(24) 

[0324] If 2 30^ A, 

[0325] S = 0...(25) 

[0326] FIG. 29 shows a relation between the saturation A and the 
saturation highlighting index S. As shown in the same fig- 
ure, the saturation highlighting index S varies gradually in 
the range from a maximum value of 50 to a minimum 
value of 0 so as to be large at a small saturation A and 
small at a large saturation A. 

[0327] Although in this embodiment there is utilized a saturation 



range occupied by a certain liigher ratio in tlie totaled sat- 
uration distribution, tiiis does not constitute any limita- 
tion. For example, a mean value or a median may be used 
in calculating the saturation highlighting index S. How- 
ever, using a certain higher ratio in the saturation distri- 
bution affords a good result as a whole because a sudden 
error becomes less influential. 

[0328] In highlighting the saturation on the basis of the satura- 
tion highlighting index S, if image data has a saturation 
parameter as described above, the said parameter may be 
transformed. However, in the case where the color specifi- 
cation space of RGB is adopted as in this example, the 
saturation highlighting can be done by once transforming 
the space into the Luv space as a standard colorimetric 
system and subsequent radial shift in the Luv space. 

[0329] First, with reference to the color transformation table, 
gradation data (Rx, Gx, Bx) of RGB is transformed into 
gradation data (L, u, v) of Luv, then the gradation data (L, 
u, v) is transformed in accordance with the following ex- 
pressions: 

[0330] u'= (S+100)/100xu...(26) 

[0331] v'= (S+100)/100xv...(27) 



[0332] As noted previously, since tlie saturation highligliting in- 
dex S ranges from a maximum value of 50 to a minimum 
value of 0, (u, v) is multiplied a maximum of 1.5 times 
into (u'.v'). Where the image data is of RGB, the grada- 
tion data (L, u',v') of Luv is then re-transformed into 
RGB, whereby this image transforming process is termi- 
nated. 

[0333] In the transformations of expressions (26) and (27) the 
saturation highlighting is performed irrespective of the 
parameter L, but such transformations are sometimes un- 
desirable. FIG. 30 shows a state in which the Luv space is 
cut vertically. As shown in the same figure, this color 
specification space is of a shape such that the bottoms of 
two cones having vertexes at L=0 and L= 100 are made 
face to face with each other. Therefore, if coordinate val- 
ues of (u, v) are shifted radially outwards for enlargement 
in accordance with the expressions (26) and (27), it is 
possible to shift the coordinate values from point Bl to 
point Bl'in a range where the luminance L is not so small, 
but in a range where the luminance L is very small, a shift 
from point B2 to point B2' results in breaking through the 
space which is in an inverted cone shape. Such a transfor- 
mation is actually infeasible and results in deviation of 



hue. 

[0334] As a countermeasure, the saturation highlighting index S 
is shifted in proportion to the luminance L To be more 
specific: 

[0335] If L < 30, 

[0336] S'= SxO = 0...(28) 
[0337] |f30^L<50, 
[0338] S'= Sx0.8...(29) 
[0339] IfSO^L, 
[0340] s'=S...(30) 

[0341] Even in the portion of a large luminance L the aforesaid 
displacement will result in breaking through a conical 
space, for which the same correction as above may be ef- 
fective. In printing with a printer, however, the image data 
of the portion with a large luminance L is presented nearly 
whitely, so there seems to be little influence of color shift. 
For this reason it turned out that the above correction 
need not be performed even for speed-up of processing. 

[0342] In the above example the saturation highlighting is per- 
formed by once transforming the image data of RGB to the 
image data in the Luv space and, after saturation high- 



lighting, re-transforming to RGB, with the result that the 
increase of the calculation volume is unavoidable. In view 
of this point, a description will be given below of a modi- 
fication in which the gradation data of RGB is utilized as it 
is in saturation highlighting. 

[0343] When component values are of hue components which are 
in a substantially equal relation like the color specification 
space of RGB, the relation of R=G=B results in gray which 
is free of saturation. Therefore, assuming that the compo- 
nent of a minimum value among the components of RGB 
merely causes the saturation to drop without influencing 
the hue of each pixel, it can be said that the saturation 
can be highlighted by subtracting such minimum value 
from all the component values and enlarging the resultant 
differential value. 

[0344] First, a saturation highlighting parameter S ratio advanta- 
geous to calculation is determined from the above satura- 
tion highlighting index S, as follows: 

[0345] s ratio = (S+ 100)/ 100.. .(31) 

[0346] In this case, when the saturation highlighting index S is 0, 
the saturation highlighting parameter S ratio becomes 
equal to 1, at which saturation is not highlighted. Next, 
assuming that the component value of blue (B) is a mini- 



mum value among the components (R,G,B) of RGB grada- 
tion data, transformation is performed as follows using 
the saturation highlighting parameter S ratio: 
[0347] R'= B+(R-B)xS ratio.. .(32) 

[0348] c'= B+(G-B)xS ratio.. .(33) 
[0349] B'=B...(34) 

[0350] As a result, the two color transformations which recipro- 
cate once between the color specification space of RGB 
and the Luv space become unnecessary, and therefore it is 
possible to shorten the calculation time. In this example 
there is adopted a method wherein a minimum value of 
component is simply subtracted from the other compo- 
nent values with respect to a saturation-free component, 
but in the subtraction of the saturation-free component 
there may be adopted another transforming expression. 
However, the transformations of the above expressions 
(32) to (34) are advantageous in that the calculation vol- 
ume is reduced because the only subtraction of a mini- 
mum value does not involve multiplication and division. 

[0351] The adoption of expressions (32) to(34) makes a satisfac- 
tory transformation possible, but in this case the high- 
lighting of saturation tends to improve luminance, result- 



ing in the whole becoming bright. In view of this point, 
the following transformation example uses a differential 
value obtained by subtracting an equivalent value of lumi- 
nance from each component value. 

[0352] First, if the color transformation is made into the above 

Luv space for the determination of luminance, the calcula- 
tion volume will become very large, so there is utilized the 
following transforming expression which determines the 
luminance directly from RGB and which is used in televi- 
sion for example as is the case with the expression (9): 

[0353] Y = 0.30R+0.59G+0.11B...(35) 

[0354] On the other hand, saturation highlighting is based on the 
following expressions: 

[0355] R'= R+DR...(36) 
[0356] G'= G +DG...(37) 
[0357] B'= B +DB...(38) 

[0358] The adjustable values □R,dG andnB are obtained as follows 
each on the basis of a differential value from the lumi- 
nance: 

[0359] nR = (R-Y)xS ratio.. .(39) 
[0360] uC = (G-Y)xS ratio.. .(40) 



[0361] nB = (B-Y)xS ratio.. .(41) 

[0362] As a result, 

[0363] R' = R + (R-Y)xS ratio ...(42) 

[0364] C'= C + (C-Y)xS ratio ...(43) 

[0365] B'= B + (B-Y)xS ratio ...(44) 

[0366] Tlius, transformation can be effected. Tliat tlie luminance 

is retained is evident from the following expressions: 

[0367] Y'= Y+DY...(45) 

[0368] nY = 0.30DR+0.59DG+0.11DB 

[0369] = s ratiox{(0.30R+0.59G+0.11B)-Y} 

[0370] = 0...(46) 

[0371] wiien tlie inputted image is gray (R=G=B), the luminance 
Y is in the relation of Y=R=G=B, so that the adjustable 
values are in the relation ofDR=DG=DB= 0, with no color 
developed in the saturation-free color. By using the ex- 
pressions (42) to (44), the luminance is retained and the 
whole of the image does not become bright even if satu- 
ration is highlighted. 

[0372] For the transformation of image data in step S214, any of 
the above methods is used and the operation of obtaining 



RGB gradation data (R',G',B') after transformation from 
the RGB gradation data of each pixel is performed for all 
the pixels. 

[0373] The above saturation highlighting process can be summa- 
rized as follows. 

[0374] Saturation values X of the pixels thinned out in step S102 
are determined and totaled into a saturation distribution, 
then it is judged in step S204 whether the inputted image 
is a binary image such as a black-and-white image or not, 
while in step S206 it is judged whether the image is a nat- 
ural picture or not. Then, except the case where the image 
is a binary image and the case where it is not a natural 
picture, it is judged in step S208 whether the image data 
is framed or not. If there is a frame portion, it is removed 
and the minimum saturation A is determined in a prede- 
termined higher range of the resulting saturation distribu- 
tion. 

[0375] Once the minimum saturation A is obtained, a saturation 
index (saturation highlighting index) S is determined from 
the range to which the saturation A belongs and in accor- 
dance with the following expressions: 

[0376] If A < 92, S = -Ax(10/92) + 50.. .(22) 

[0377] If 92^A < 184, S = -Ax(10/46) + 60.. .(23) 



[0378] If 184^A < 230, S = -Ax(10/23) + 100.. .(24) 
[0379] If 230^A, S = 0...(25) 

[0380] On the basis of the saturation highlighting index S thus 
obtained, there is made transformation of image data in 
step S214. As an example, if the gradation data of RGB is 
utilized directly while retaining luminance, the transfor- 
mation of image data is performed for all the pixels in ac- 
cordance with the following expressions: 

[0381] R'=R + (R-Y)xS ratio ...(42) 

[0382] G ' = G + (G-Y)xS ratio ...(43) 
[0383] B'= B + (B-Y)xS ratio ...(44) 

[0384] By SO doing, even in the case of a photograph with weak 
saturation, it is possible to highlight saturation which 
varies in a narrow range, to obtain a vivid image. There- 
after, if the image data is outputted as RGB to the display 
32, a vivid image will be reproduced on the screen, and if 
it is outputted to the printer 31, a vivid image will be re- 
produced on paper by transformation to the CMYK color 
specification space of color ink and subsequent gradation 
transformation. 

[0385] Needless to say, as noted previously, such processing is 



not applied to a binary image or an image whicli is not a 
natural picture. Although in the above example the condi- 
tions for selecting the saturation highlighting index is 
made constant, a modification may be made so that the 
user can select it through a predetermined GUI on the 
computer 21. By so doing, transformation is made auto- 
matically so as to give an optimum range on the basis of 
the value set by the user. Particularly, without transforma- 
tion to the Luv space, hues may not always be retained. In 
this case some measure maybe taken so that the shift of 
hue can be ignored. For example, the saturation high- 
lighting index S is weakened to prevent rotation of the 
coloring matters. Further, it is suggested to make the 
aforesaid plural transforming methods selectable or pro- 
vide optimum settings of saturation highlighting index for 
those methods. It is also possible for the user to desig- 
nate part of image data so that the saturation highlighting 
process is executed only within the range concerned. 
[0386] If the expressions (42) to (44) are utilized in the saturation 
highlighting process, luminance is retained and the whole 
does not become bright even when saturation is high- 
lighted. In the case of highlighting only saturation in the 
above manner, there arises no problem, but when the 



contrast enlarging process and the saturation liigliligliting 
process are executed simultaneously, the resulting syner- 
gistic effect will make the image loud. 

[0387] Jo avoid this inconvenience, a correlation may be taken to 
suppress both processings relative to each other. FIGS. 31 
to 33 illustrate a contrast enlarging-saturation highlight- 
ing interadjusting process, of which FIG. 31 shows a main 
processing in the said interadjusting process, and FIGS. 32 
and 33 show the contrast enlarging process and the satu- 
ration highlighting process each individually in more de- 
tail. The details of these processings are the same as 
those described above and are therefore omitted here. 

[0388] In step S3 10 there are performed thinning-out and total- 
ing of pixels, then in steps S320 and S330 which are front 
stages of both contrast enlarging process and saturation 
highlighting process there are obtained a contrast enlarg- 
ing coefficient 'a' and a saturation highlighting coefficient 
S, and thereafter in step S340 which corresponds to a 
medium stage there is executed a highlighting suppress- 
ing process to determine a formal saturation highlighting 
coefficient S'. Thus, in this example, the contrast enlarg- 
ing coefficient 'a' is first determined and with this coeffi- 
cient fixed there is performed the highlighting suppress- 



ing process to determine the formal saturation liigliliglit- 
ing coefficient S'. 

[0389] The contrast enlarging coefficient 'a' and the formal satu- 
ration highlighting coefficient S'are correlated with each 
other in the following manner: 

[0390] S'= Sx(l/a)...(47) 

[0391] As mentioned previously, the contrast enlarging coeffi- 
cient 'a' is 1 or larger and (1/a) becomes smaller than 1 as 
the enlargement tendency becomes stronger. Conse- 
quently, the formal saturation highlighting coefficient 
S' becomes smaller than the temporary saturation high- 
lighting coefficient S determined in the above manner. 
That is, the parameter used in the saturation highlighting 
process is made small by the parameter used in the con- 
trast enlarging process to suppress the highlighting pro- 
cess. 

[0392] In this example, as shown in FIG. 34, a correlation is taken 
such that the saturation highlighting coefficient S is weak- 
ened by the contrast enlarging coefficient 'a.' In this sense 
it can be said that a highlighting process suppressing fil- 
ter is provided. However, as shown in FIG. 3 5, it is also 
possible to make a correlation so that the contrast enlarg- 
ing coefficient 'a' is weakened by the saturation highlight- 



ing coefficient S. Furtlier, a correlation may be made as 
sliown in FIG. 36 in which both coefficients are weakened 
relative to each other. Not a simple filter but a transfor- 
mation table may be prepared for reference to concrete 
values of the contrast enlarging coefficient 'a' and the sat- 
uration highlighting coefficient S, as shown in FIGS. 37(A) 
and (B). Such a transformation table permits a more de- 
tailed setting. 

[0393] There also is the case where it is not necessary to weaken 
both coefficients relative to each other. For example, 
when the contrast enlarging coefficient 'a' and the satura- 
tion highlighting coefficient S are small, both may act 
each individually and may not exert a synergistic bad in- 
fluence. In such a case, there may be provided a non- 
interference area as in FIG. 38 so that both coefficients are 
subject to a weakening influence upon reaching respective 
certain threshold values. It is also possible to let one co- 
efficient act on the other so as to weaken the other upon 
reaching a certain threshold value or larger. 

[0394] After the highlighting process has been suppressed as 

above by using the operation expression (47) or using the 
transformation table, there lastly is performed in step 
S3 50 the transformation of image data with use of the 



foregoing highlighting coefficients. More specifically, in 
correspondence to the rear stage of each of the above 
processings, data transformation is performed pixel by 
pixel with respect to the image data of each attentional 
pixel. 

[0395] Turning back to the flowchart of FIG. 7, after the end of 

both contrast enlarging process and saturation highlight- 
ing process as described above, an edge highlighting pro- 
cess is executed in step SI 14, which process is illustrated 
in FIG. 39. 

[0396] In the edge highlighting process, the number of pixels in 
the width direction of the inputted image and that in the 
height direction are multiplied together to obtain the 
number of pixels in step S402. Then, in step S404 there is 
determined an edge highlighting degree proportional to 
the thus-detected number of pixels. The edge highlight- 
ing degree depends greatly on the edge highlighting 
method, so reference will first be made to the edge high- 
lighting method. In this example there is used such an 
unsharp mask 40 as shown in FIG. 40. In the unsharp mask 
40, the central value '100' is used as a weight of an atten- 
tional pixel PiJ in a matrix-like image data and the 
marginal pixels are weighted correspondingly to the nu- 



merical values described in tlie squares of the maslc and 
are utilized for integration, which is performed in accor- 
dance with the following expression (48) if the unsharp 
mask 40 is used: 
[0397] p'ij = (i/632)I(MijxPij) ...(48) 

[0398] ij 

[0399] In expression (48), '632' stands for a total value of 

weighting coefficients, Mij stands for a weight coefficient 
described in each square of the unsharp mask, and PiJ 
stands for image data of each pixel, in which iJ represents 
coordinate values of columns and rows. 

[0400] Given that the image data after edge highlighting is as- 
sumed to be QiJ, the edge highlighting calculation is per- 
formed in accordance with the following expression (49): 

[0401] Qij = pij + Cx{Pij-P'ij} ...(49) 

[0402] The meaning of the expression (49) is as follows. P'ij 
stands for the result of addition at a low weight of the 
marginal pixels relative to the attentional pixel and hence 
it is an unsharp image data, which is the same as image 
data having passed through a low- pass filter. Therefore, 
'Pij-P'ij' represents the result of having subtracted a low 
frequency component from all the components, indicating 



the same image data as that having passed through a 
high-pass filter. Then, if this high frequency component 
from the high-pass filer is multiplied by an edge high- 
lighting coefficient C and the result is added to Pij, it fol- 
lows that the high frequency component has been in- 
creased in proportion to the edge highlighting coefficient. 
The edge is highlightened in this way. 

[0403] Also in this sense the edge highlighting degree can be 

changed by the edge highlighting coefficient C. Therefore, 
if the number of pixels is large, the edge highlighting co- 
efficient C is increased, while if the number of pixels is 
small, the coefficient C is decreased. In an image width 
and height condition of widthxheight, E ratio is repre- 
sented by the following expression (50): 

[0404] E ratio = min(width, height)/640+l...(50) 

[0405] For the E ratio thus obtained, the edge highlighting coeffi- 
cient C was determined as follows: 
[0406] For E ratio < 1, C=l 

[0407] For l^E ratio < 3, C=2 

[0408] For 3^E ratio, C=3 

[0409] To be more specific, the edge highlighting coefficient C is 
1 if the number of shorter pixels is less than 640, C is 2 if 



the said number is 640 or more and less tlian 1920, and C 
is 3 if the said number is 1920 or more. Although the 
edge highlighting coefficient C is set as above in this ex- 
ample, it may be varied in a proportional manner because 
the image size may change in some particular dot density. 

[0410] Since the edge highlighting degree varies also according 
to the size of the unsharp mask, a large size of an un- 
sharp mask can be used for a large number of pixels, or a 
small size of an unsharp mask for a small number of pix- 
els. Needless to say, both edge highlighting coefficient C 
and unsharp mask 40 may be changed, or the edge high- 
lighting degree may be changed with respect to only one 
of the two. As is seen also from the figure, the unsharp 
mask 40 is most heavily weighted at the central portion 
and gradually becomes smaller in the values of weighting 
toward the marginal portion. This degree of change is not 
always fixed but may be changed as necessary. 

[0411] In the unsharp mask 40 of 7x7 shown in FIC.40, the 

weighting value of the outermost peripheral squares is 0 
or 1. As to 0, a weighting multiplication is meaningless, 
and the weighting of 1 has only a very slight weight in 
comparison with the total square value of 632. In this ex- 
ample, therefore, such an unsharp mask 41 of 5x5 as 



shown in FIG. 41 is used as a substitute for the unsharp 
mask 40 of 7x7. The unsharp masl< 41 corresponds to an 
unsharp mask obtained by omitting the outermost pe- 
riphery of the 7x7 unsharp mask 40. Both are coincident 
with each other. At the inside 5x5 mask portion the un- 
sharp mask 40 coincides in weighting with the unsharp 
mask 41. As a result, a concrete processing volume is re- 
duced by half. 

[0412] In this example, moreover, the operation volume is re- 
duced in the following manner. In the case where the gra- 
dation data of RGB is to be processed, each component 
value corresponds to the luminance (lightness) of the as- 
sociated color component. Therefore, the calculation of 
the expression (49) should be performed individually for 
each gradation data of RGB. However, if multiplication and 
addition are repeated by the number to times correspond- 
ing to the number of the squares of the unsharp mask 40, 
such an individual operation for each component in- 
evitably results in a large operation volume. 

[0413] On the other hand, it can be said that such edge high- 
lighting can be done by changing luminance while retain- 
ing hue. It follows that the processing volume is de- 
creased by making calculation for the luminance obtained. 



not for the component values of RGB. If the foregoing 
transforming expression of luminance Y is used, the ex- 
pressions (48) and (49) can be rewritten as follows: 
[0414] Y'ij = (l/632)I(MijxYij) ...(51) 

[0415] ij 

[0416] Y'ij = Yij + C {Yij-Y'ij) ...(52) 

[0417] Further, Yij-Y'ij may be rewritten as follows: 

[0418] delta = Yij-Y'ij. ..(53) 

[0419] R',c'and B'after transformation can be calculated as fol- 
lows: 

[0420] R'=R + deltaxC 
[0421] G'=G + deltaxC 
[0422] B'= B + deltaxC ...(54) 

[0423] As a result, the number of times of multiplication and ad- 
dition is reduced to one third, so that the total processing 
time can be diminished about 50d70%. According to the 
results of the transformation, there no longer was any 
highlighting of color noise and the image quality was im- 
proved. 

[0424] The edge highlighting process corresponds to a process 



wherein image data (R'G'B') after edge liighlighting is 
calculated using the unsharp mask 40 with respect to 
each pixel in the matrix-like image data. In steps S406 to 
S412 is represented a loop processing wherein the edge 
highlighting process is repeated for each pixel. In step 
S410 included in the loop processing, an attentional pixel 
is shifted successively in both horizontal and vertical di- 
rections, and this processing is repeated until when the 
pixel being processed is Judged to be the final pixel in 
step S412. 

[0425] In the operation of expression (48), however, both multi- 
plication and addition are needed by the number of the 
squares of the unsharp mask 40 for the pixels located 
around the attentional pixel, resulting in that the process- 
ing volume is very large. On the other hand, the portion 
which requires edge highlighting is the edge portion of 
the inputted image and is therefore limited to the portion 
where image data are markedly different between adjacent 
pixels. Under the circumstances, it can be said that opera- 
tion may be made only in the case of a great difference in 
image data between adjacent pixels. 

[0426] Jo be more specific, in step S406 there is made a com- 
parison between adjacent pixels with respect to image 



data and only when the resulting difference is large, the 
operation using the unsharp mask in step S408 is con- 
ducted. By so doing, in most of the image data portion, 
not the edge portion, it is no longer required to perform 
the operation using the unsharp mask and thus the pro- 
cessing volume is decreased markedly. 
[0427] As shown in FIG. 42, the above comparison is required 

eight times because there are eight pixels around the at- 
tentional pixel. However, when there is made a compari- 
son, for example, between the attentional pixel located at 
the center and the pixel of square "5," it is seen that when 
the attentional pixel was previously the square '5' there 
was made a comparison once with the pixel of square '1.' 
Therefore, assuming that it is sufficient for adjacent pixels 
to be compared only once, any one alone is sufficient for 
comparison with respect to the combinations opposite in 
direction of 1 and 5, 2 and 6, 3 and 7, and 4 and 8. Com- 
parison is made in four directions in the said combina- 
tions. 

[0428] In this way, as to a natural picture using a large number of 
colors, the enlargement of contrast is performed auto- 
matically within an optimum range, and saturation high- 
lighting and edge highlighting are also conducted auto- 



matically at a highlighting degree best suited for the im- 
age concerned. Needless to say, in the case of an image 
which is not a natural picture, such processings are 
skipped as unnecessary processings. 

[0429] Although an optimum image processing has been con- 
ducted above in the application 2 Id on the basis of the 
type of image data detected from the number of colors 
used, also in the printing process an optimum processing 
is also selected according to the type of image detected. 
FIG. 43 is a flowchart showing a processing corresponding 
to the printing process. In step S502 there is formed a 
raster data, then in step S504 the type of image data is 
judged on the basis of the foregoing flag, and any one of 
steps S506 and S508 as color transformation processings 
is executed. Thereafter, binary coding is performed in 
step S510 and printing data is outputted in step S512. 

[0430] It is a color transformation processing as a combination of 
both interpolating operation processing and cache pro- 
cessing that is carried out in step S506. The procedure 
thereof is shown in FIG. 44 et seq. In the interpolating op- 
eration shown in the same figure there is performed not 
only an 8-point interpolating operation involving substan- 
tial operations but also caching for color transformation 



without any substantial operation. 
[0431] As to the 8-point interpolating operation, its principle is 
shown in FIG. 45. A cube comprising eight lattice points 
around coordinates P involving RGB gradation data as 
component values in a color specification space before 
transformation is here assumed. Given that the transfor- 
mation value at kth vertex of the cube is Dk and the vol- 
ume of the cube is V, a transformation value Px at point P 
of the cube can be interpolated as follows from a weight 
based on the ratio of volume Vk of such eight small rect- 
angular parallelepipeds as shown in the figure which are 
divided at point P: 

[0432] 8 

[0433] Px =I(Vk/V)DDk...(55) 

[0434] k=l 

[0435] Therefore, eight lattice points which surround the coordi- 
nates are specified and calculation is performed for each 
of GMY gradation data at each lattice point. 

[0436] vvith reference to the flowchart of FIG. 44 shows that in the 
processing routine of this interpolating operation the said 
8-point interpolating operation is not always carried out, 
but there is performed caching for omitting the 8-point 



interpolating operation wliile mal<ing reference to such a 
cacliing table as shown in FIG. 46. This caching table is a 
table of a certain capacity for retaining CMY gradation 
data obtained by executing the8-point interpolating oper- 
ation using the RGB gradation data before transformation. 
Initially, this table is blank, but the GMY gradation data 
obtained just after execution of the 8-point interpolating 
operation in step S606 is added and updated in step 
S608. When the processing routine of 'cache + interpolat- 
ing operation' is executed in step S506 as shown in FIG. 
43, the interior of the cache table is retrieved using the 
gradation data of RGB before transformation as compo- 
nent values in the first step S602, and upon cache hit 
(discovery by the retrieval), reference is made in step S612 
to the GMY gradation data stored. 
[0437] In the color transformation processing, the processing is 
repeated until the final image is obtained in step S610 for 
transformation of the RGB gradation data into GMY grada- 
tion data with respect to each of the dot matrix-like pix- 
els. 

[0438] Although such an interpolating operation is large in the 
operation volume, but since the number of colors used is 
small in the case the inputted image is not a natural pic- 



ture, the result of a single operation can be utilized re- 
peatedly. Therefore, when it is judged that the type of im- 
age data is not a natural picture, it can be said that the 
processing 'cache + interpolating operation' is very effec- 
tive. 

[0439] Next, reference will be made below to the pre-gray level 
transformation of step S508 which is another color trans- 
formation processing. 

[0440] The flowchart of FIG.47 and the diagram of FIG.48 show- 
ing an error distribution of pixel are for explaining an 
outline of the pre-gray level transformation. Since the ba- 
sic expression of the foregoing 8-point interpolation re- 
quires multiplication eight times and addition seven 
times, resources and time consumption are large in both 
hardwarization and execution using software. For easier 
color transformation, therefore, the applicant in the 
present case has developed a gray level transformation as 
a substitute for the interpolating operation in Japanese 
Patent Laid Open No.30772/95. 

[0441] The pre-gray level transformation disclosed in the above 
unexamined publication uses an error diffusing method 
for example for gray level transformation of pixel grada- 
tion data into coincidence with lattice coordinates. Lattice 



coordinates proximate to tlie pixel to be transformed are 
searched for (S702), then an error (dg) from the lattice co- 
ordinates is calculated (S704), and the error (dg) is dis- 
tributed to nearby pixels (S706). This is all of the process- 
ing. Thus, the burden of the operation can be lightened to 
a great extent in comparison with the repetition of multi- 
plication and addition. As to the reason why high accuracy 
can be maintained without interpolating operation, it is 
fully stated in the above unexamined publication, so a de- 
tailed description thereof is here omitted. Since a gray 
level transformation for binary coding is performed even 
after color transformation, the gray level transformation 
conducted first is called herein a pre-gray level transfor- 
mation. 

[0442] Also in the pre-gray level transformation, gray level trans- 
formation is conducted for each of rasterized dot matrix- 
like pixels and therefore the processing is repeated until 
the final pixel in step S708. In the subsequent step S710, 
reference is made to the color transformation table using 
the RGB gradation data after gray level transformation. At 
this time, because of lattice points, it is not necessary to 
conduct an interpolating operation and thus the reading 
process is very easy. 



[0443] In the case where the number of colors used is very large 
like a natural picture, the probability of hit is low even in 
the foregoing processing of cache + interpolating opera- 
tion, so that the number of interpolating operation in- 
creases. In such a pre-gray level transformation it is nec- 
essary to execute such a processing as error diffusion for 
all the pixels, but the processing volume as a whole is 
small because the processing conducted each time is sim- 
ple. Therefore, with respect to image data using many 
colors and having been Judged to be a natural picture, the 
entire processing volume can be reduced by selecting a 
suitable color transformation processing based on such 
pre-gray level transformation. 

[0444] In the embodiment being considered there is adopted the 
color transformation processing of cache + interpolating 
operation for an unnatural picture small in the number of 
colors used, while for a natural picture using many colors 
there is adopted a color transformation processing based 
on the pre-gray level transformation. However, no limita- 
tion is placed on such a combination. It is also possible to 
adopt another combination. In other words, a color trans- 
formation processing most suitable for the type of in- 
putted image may be selected freely and utilized auto- 



matically. 

[0445] Thus, by totaling luminance values of pixels which have 
been selected by a thinning-out processing for their im- 
age data, to obtain a luminance distribution, (step S102), 
it becomes possible to count the number of colors used 
(step S104). As a result, if the number of colors used is 
large, the related image data can be judged to be a natu- 
ral picture. Besides, it becomes possible to automatically 
select a contrast enlarging process (step SI 10), a satura- 
tion highlighting process (S112)and an edge highlighting 
process (SI 14) which are suitable for application to the 
natural picture, on the basis of the said result of Judg- 
ment. Also in the printing process, a color transformation 
based on the pre-gray level transformation is performed if 
the image data concerned is a natural picture (S508), 
while if the image data concerned is not a natural picture, 
there is performed the color transformation of 'cache + 
interpolating' operation (S506), and thus it is possible to 
automatically select a color transformation processing of a 
small processing volume. 



